-- Logs begin at Sat 2022-10-22 23:29:14 UTC, end at Sun 2022-10-23 00:38:23 UTC. -- Oct 22 23:54:26.757207 dv-cinder-861566-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 22 23:54:26.768765 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 22 23:54:26.770731 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Oct 22 23:54:26 2022] *** Oct 22 23:54:26.770731 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 22 23:54:26.770731 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 22 23:54:26.770731 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: nodename: dv-cinder-861566-5-iscsi Oct 22 23:54:26.770731 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: machine: x86_64 Oct 22 23:54:26.770731 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: clock source: unix Oct 22 23:54:26.770731 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: pcre jit disabled Oct 22 23:54:26.770731 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: detected number of CPU cores: 4 Oct 22 23:54:26.770731 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: current working directory: / Oct 22 23:54:26.770731 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: detected binary path: /usr/bin/uwsgi-core Oct 22 23:54:26.771389 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: your processes number limit is 47680 Oct 22 23:54:26.771389 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: your memory page size is 4096 bytes Oct 22 23:54:26.771389 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: detected max file descriptor number: 2048 Oct 22 23:54:26.771389 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: lock engine: pthread robust mutexes Oct 22 23:54:26.771389 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: thunder lock: enabled Oct 22 23:54:26.771389 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 22 23:54:26.771389 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 22 23:54:26.809820 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: Python main interpreter initialized at 0x56392314e9c0 Oct 22 23:54:26.809820 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: python threads support enabled Oct 22 23:54:26.809820 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: your server socket listen backlog is limited to 100 connections Oct 22 23:54:26.809820 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: your mercy for graceful operations on workers is 90 seconds Oct 22 23:54:26.810148 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: mapped 403077 bytes (393 KB) for 2 cores Oct 22 23:54:26.810228 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: *** Operational MODE: preforking *** Oct 22 23:54:26.810487 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: *** uWSGI is running in multiple interpreter mode *** Oct 22 23:54:26.810487 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: spawned uWSGI master process (pid: 81370) Oct 22 23:54:26.810664 dv-cinder-861566-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 22 23:54:26.813169 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: spawned uWSGI worker 1 (pid: 81380, cores: 1) Oct 22 23:54:26.813169 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: spawned uWSGI worker 2 (pid: 81381, cores: 1) Oct 22 23:54:26.813169 dv-cinder-861566-5-iscsi devstack@keystone.service[81370]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 22 23:54:31.113636 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 22 23:54:31.113636 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: from cryptography.utils import int_from_bytes Oct 22 23:54:31.114344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 22 23:54:31.114344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: from cryptography.utils import int_from_bytes Oct 22 23:54:31.114777 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 22 23:54:31.114777 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: from cryptography.utils import int_from_bytes Oct 22 23:54:31.114902 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 22 23:54:31.114902 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: from cryptography.utils import int_from_bytes Oct 22 23:54:31.163991 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 22 23:54:31.164192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 22 23:54:31.164321 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 22 23:54:31.164409 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] command line args: [] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 22 23:54:31.164473 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 22 23:54:31.164651 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 22 23:54:31.164755 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] command line args: [] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 22 23:54:31.164824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ================================================================================ {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 22 23:54:31.165003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 22 23:54:31.165091 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] admin_token = **** {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.165206 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ================================================================================ {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 22 23:54:31.165298 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] config_dir = [] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.165468 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.165554 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] admin_token = **** {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.165667 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] config_source = [] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.165752 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] config_dir = [] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.165838 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] debug = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.165984 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.166071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.168371 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] config_source = [] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.168371 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] debug = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.168535 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.168535 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.168672 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.168762 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.168825 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.168825 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.168943 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.168943 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] list_limit = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.169059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.169123 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] log_config_append = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.169182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.169240 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.169240 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] log_dir = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.169361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] list_limit = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.169420 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] log_file = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.169483 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] log_config_append = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.169542 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.169605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.169674 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.169734 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] log_dir = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.169734 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] log_file = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.169847 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.169847 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.169960 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.170026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.170092 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.170155 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.170155 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.170267 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.170334 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.170400 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.170466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.170538 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.170538 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.170656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.170656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.170777 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.170837 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.170895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.170952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.171009 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.171074 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.171134 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.171197 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.172056 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.172181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.172390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.172635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.172796 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] publish_errors = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.172796 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.173134 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.173313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.173465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.173637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.173637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.173972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.174156 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.174312 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] publish_errors = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.174489 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] standard_threads = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.174489 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.174835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.175013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.175174 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.175346 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.175523 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.175700 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] use_journal = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.175700 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] use_json = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.176000 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.176177 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] use_stderr = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.176348 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.176543 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] use_syslog = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.176746 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] standard_threads = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.176908 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.177028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.177148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.177148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.177411 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.177411 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.177639 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.177761 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] use_journal = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.177876 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.178019 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] use_json = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.178135 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.178257 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] use_stderr = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.178412 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.178530 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] use_syslog = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.178650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.178766 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 22 23:54:31.178880 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.179003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.179120 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.179240 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.179386 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.179538 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.179654 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.179654 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.179881 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.179997 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.180111 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.180231 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.180353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.180476 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.180609 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.180723 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.180848 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.180965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.181081 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.181206 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.181323 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.181448 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.181448 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.181670 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.181784 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.181906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.182022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.182137 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.182250 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.182364 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.182477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.182668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.182791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.182908 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.183027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.183027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.183358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.183498 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.183620 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.183735 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.183857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.183971 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.184093 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.184217 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.184332 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.184461 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.184461 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.184771 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.184895 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.185012 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.185132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.185249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.185363 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.185482 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.185610 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.185610 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.185826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.185946 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.186061 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.186176 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.186299 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.186457 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.186577 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.186692 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.186806 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.186919 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.187044 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.187161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.187161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.187375 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.187494 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.187609 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.187734 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.187851 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.187964 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.188078 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.188499 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.188621 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.188823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.188948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.189065 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.189065 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.189279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.189397 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.189397 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.189629 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.189751 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.189872 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.189987 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.190101 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.190226 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.190342 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.190458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.190572 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.190685 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.190799 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.190907 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.191005 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.191100 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.191195 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.191291 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.191291 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.191476 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.191583 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.191687 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.191814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.191923 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.192041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.192165 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.192282 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.192386 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.192543 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.192675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.192793 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.192906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.193022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.193137 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.193252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.193367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.193490 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.193614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.193743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.193861 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.193976 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.193976 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.194168 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.194282 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.194405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.194513 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.194627 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.194743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.194846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.194966 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.195093 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.195210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.195325 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.195448 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.195565 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.195681 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.195793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.195910 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.196025 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.196129 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.196232 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.196347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.196347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.196600 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.196718 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.196834 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.196955 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.197073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.197196 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.197314 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.197429 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.197544 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.197659 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.197774 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.197897 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.198014 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.198136 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.198252 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.198367 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.198520 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.198647 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.198765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.198882 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.198997 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.199112 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.199227 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.199353 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.199471 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.199594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.199711 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.199826 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.199942 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.200134 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.200253 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.200369 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.200522 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.200641 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.200782 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.200909 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.201039 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.201157 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.201273 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.201388 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.201511 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.201628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.201751 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.201869 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] auth.external = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.201987 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.202102 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.202217 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.202332 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.202469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.203092 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.203208 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.203324 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] auth.password = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.203439 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.203554 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] auth.token = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.203677 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.203794 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.203910 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.204024 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.204140 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.204255 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.204404 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.204551 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.204683 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.204801 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.204918 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.205041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.205161 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.205277 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.205401 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.205401 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.205696 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] credential.caching = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.205822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.205938 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.206054 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.206179 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.206298 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.206415 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.206531 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.206647 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.206763 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] auth.external = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.206890 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.207009 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.207137 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.207257 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.207375 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.207491 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.207615 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.207739 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] auth.password = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.207856 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.207973 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] auth.token = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.208089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.208205 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.208205 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.208443 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.208632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.208751 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.208868 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.208984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.209113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.209233 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.209361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.209479 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.209602 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.209720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] federation.caching = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.209845 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] credential.caching = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.209963 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.210079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.210196 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.210312 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.210429 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.210554 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.210673 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.210789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.210905 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.211023 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.211140 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.211265 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.211383 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.211501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.211617 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.211734 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.211850 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.211975 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.212094 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.212094 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.212388 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.212523 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.caching = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.212654 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.212775 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.212892 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.213008 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.213124 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.213240 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.213363 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.213483 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.213600 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.213717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.213833 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] federation.caching = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.213955 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.214082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.214200 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.214317 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.214433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.214559 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.214683 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.214877 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.214998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.215114 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.215231 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.215355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.215474 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.215598 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.215717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.215833 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.215955 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.216073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.216196 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.216321 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.216440 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.216602 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.216721 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.216838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.caching = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.216838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.217068 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.217188 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.217304 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.217421 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.217537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.217653 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.217779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.217897 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.218019 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.218137 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.218940 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.219042 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.219185 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.219347 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.219471 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.219587 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.219711 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.219830 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.219955 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.220074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.220189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.220305 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.220421 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.220551 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.220682 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.220801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.220917 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.221032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.221147 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.221261 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.221385 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.221503 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.221619 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.221734 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.221848 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.221964 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.222087 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.222204 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.222310 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.222408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.222505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.222602 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.222726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.222844 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.222961 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.223076 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.223076 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.223300 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.223418 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.223533 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.223648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.223763 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.223878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.224002 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.224002 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.224221 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.224336 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.224452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.224452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.224705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.224705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.224927 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.225045 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.225161 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.225277 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.225407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.225528 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.225644 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.225761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.225877 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.225993 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.226120 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.226239 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.226355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.226472 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.226587 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.226704 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.226828 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.226954 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.227071 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.227188 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.227305 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.227421 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.227546 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.227665 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.227781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.227898 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.228014 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.228137 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.228264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.228383 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.228545 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.228665 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.228782 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.228936 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.229096 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.229218 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.229339 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.229457 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.229574 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.229690 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.229815 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.229935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.230051 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.230168 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.230285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.230402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.230527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.230652 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.230770 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.230887 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.230887 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.231105 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.231230 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.231350 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.231350 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.231570 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.231570 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.231788 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.231788 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.232016 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.232132 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.232248 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.232364 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.232481 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.232631 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.232751 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.232868 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.232984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.233101 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.233217 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.233345 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.233465 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.233581 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.233697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.233847 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.233977 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.234175 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.234301 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.234419 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.234544 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] resource.caching = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.234663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.234780 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.234906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.235025 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.235142 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.235294 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.235415 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.235534 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.235534 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.235764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.235881 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.235998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.236115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.236231 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.236357 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.236475 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.236646 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.236763 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.236880 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] role.caching = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.236997 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.237127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] role.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.237247 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.237364 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.237481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.237596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.237714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.237848 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.237968 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.238084 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.238201 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.238318 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.238506 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.238627 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.238745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] resource.caching = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.238870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.239021 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.239140 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.239266 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.239388 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.239507 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.239624 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.239742 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.239859 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.239984 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.240103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.240221 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.240337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.240454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.240582 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.240713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.240713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.240936 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.241053 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] role.caching = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.241169 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.241286 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] role.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.241411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.241531 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.241648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.241765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.241882 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.241999 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.242125 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.242244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.242361 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.242478 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.242594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.242712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.242837 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.242958 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.243074 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.243191 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.243308 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.243308 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.243536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.243655 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.243779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.243897 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.244015 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.244132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.244257 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.244377 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.244549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.244669 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.244786 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.244915 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.244915 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.245139 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.245256 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] token.caching = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.245372 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.245489 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.245617 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.245780 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.245901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.246019 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.246135 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.246251 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.246376 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.246496 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.246613 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.246729 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.246846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.246963 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.247089 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.247208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.247325 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.247441 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.247558 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.247676 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.247800 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.247919 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.248035 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.248153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.248270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.248386 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.248533 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.248655 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.248771 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.248888 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.249049 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.249181 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.249301 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.249454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.249575 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.249692 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.connection = **** {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.249810 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] token.caching = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.249926 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.250043 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.250167 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.250288 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.250420 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.250614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.250763 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.250882 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.251009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.251128 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.251247 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.251365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.251483 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.251600 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.251725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.251845 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.251963 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.252080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.252197 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.252315 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.252445 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.252584 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.252702 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.252819 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.252937 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.253055 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.253185 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.253305 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.253424 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.253541 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.253541 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.253780 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.253899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.254016 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.254141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.connection = **** {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.254260 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.254377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.254495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81380) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 22 23:54:31.254613 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.254613 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.254613 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.254613 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.254613 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.254613 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.254613 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.254613 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.254613 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.255699 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.255699 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.255699 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.255699 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.255699 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.255699 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.255699 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.255699 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 22 23:54:31.255699 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 22 23:54:31.280223 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.280448 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.280753 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.283991 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.284229 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.284461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.289606 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.292839 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.318090 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.318318 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.320351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.320634 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.371883 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.372080 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.421721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.421938 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.422075 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.422219 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.576149 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.576405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.576619 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.576844 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.576991 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.577118 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.577237 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.577355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.577477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.577477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.577711 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.577914 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.601139 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.601301 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.601429 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.601569 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.601693 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.601811 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.601928 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.602045 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.602170 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.602289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.602406 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.602521 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.602638 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.602763 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.614287 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.614438 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.627654 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.627835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.628055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.628201 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.628326 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.628444 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.628596 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.628717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.628851 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.628975 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.629095 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.629214 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.629214 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.629447 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81380) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 22 23:54:31.645745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.645961 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 22 23:54:31.646132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.646255 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 22 23:54:31.646374 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.646508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 22 23:54:31.646631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.646750 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 22 23:54:31.646868 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.647020 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.647155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.647277 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.647536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.647663 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.647784 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.647910 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.657769 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.657940 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.658066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.658221 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.658345 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.658479 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.658602 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.658764 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.665562 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.665739 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.668462 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.668655 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.668850 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.668974 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.669139 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.669270 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.669426 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.669552 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.669712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.669835 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.669978 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.670109 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.670252 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.670376 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.670533 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.670656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.670807 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.670930 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.671085 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.671208 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.671924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.672054 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.694743 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.694884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.696201 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.696324 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.696518 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.696661 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.701155 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.701444 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.701735 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.702979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.703268 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.703556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.708086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.708377 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.709926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.710223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.714007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.714313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.714437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.714606 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.714731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.714894 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.715019 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.715171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.715295 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.715449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.715581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.715704 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.715824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.715980 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.716104 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.716251 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.732323 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.732462 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.733649 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.733773 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.737375 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.738000 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.740634 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.740937 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.741236 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.741541 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.745156 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.745765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.748365 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.749056 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.751698 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.752006 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.752296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.752419 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.752610 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.752736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.752896 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.753021 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.753189 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.753312 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.753494 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.753782 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.769004 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.769297 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.769565 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.769688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.769818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.769986 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.770262 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.770535 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.776675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.777439 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.778210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.778983 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.779760 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.780048 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.780550 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.781455 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.783400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.783689 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.784000 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.784218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.784348 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.784501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.784650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.784921 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.785215 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.785513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.795720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.796012 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.796382 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.796660 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.796786 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.797004 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.797130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.797370 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.797495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.797655 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.797790 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.798084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.798371 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.798658 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.813813 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.814999 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.816745 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.817949 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.819624 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.820858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.821326 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.822313 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.822823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.823118 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.823516 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.823932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.824232 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.824541 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.830178 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.830993 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.831643 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.832512 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.834558 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.834959 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.835249 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.835518 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.835642 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.835801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.836089 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.836371 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.843887 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.844193 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.844473 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.844763 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.845060 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.845348 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.851365 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81380) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.851653 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.851937 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.852217 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.852508 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.852791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.853065 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.853202 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 22 23:54:31.853351 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.853476 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.853623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.853747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.853903 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.854038 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.854169 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.854291 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.854419 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.854542 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.854684 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.854840 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.854972 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81380) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.855173 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.855370 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.855528 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.855803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.856078 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.856358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.856648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 22 23:54:31.889247 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56392314e9c0 pid: 81380 (default app) Oct 22 23:54:31.890181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56392314e9c0 pid: 81381 (default app) Oct 22 23:54:31.892518 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada509c-cf24-4ee1-af66-67fa7aeda023 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:31.892887 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada509c-cf24-4ee1-af66-67fa7aeda023 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:31.893168 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada509c-cf24-4ee1-af66-67fa7aeda023 None None] PATH_INFO: `/v3` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:31.897298 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Sat Oct 22 23:54:31 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:54:38.684079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee60593-d4fc-4c8f-bb79-2ee03bcc883c None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:38.684620 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee60593-d4fc-4c8f-bb79-2ee03bcc883c None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:38.684726 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee60593-d4fc-4c8f-bb79-2ee03bcc883c None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:38.689224 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:54:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:54:38.694511 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f952b-191c-4b02-9855-585f14f3ac2f None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:38.694989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f952b-191c-4b02-9855-585f14f3ac2f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:38.695082 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f952b-191c-4b02-9855-585f14f3ac2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:38.765852 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: INFO dbcounter [None req-980f952b-191c-4b02-9855-585f14f3ac2f None None] Registered counter for database keystone Oct 22 23:54:38.785174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG oslo_db.sqlalchemy.engines [None req-980f952b-191c-4b02-9855-585f14f3ac2f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81380) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 22 23:54:38.791281 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writer thread running {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 22 23:54:38.969831 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG passlib.handlers.bcrypt [None req-980f952b-191c-4b02-9855-585f14f3ac2f None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81380) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 22 23:54:38.970130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG passlib.handlers.bcrypt [None req-980f952b-191c-4b02-9855-585f14f3ac2f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81380) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 22 23:54:39.027442 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-980f952b-191c-4b02-9855-585f14f3ac2f None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:54:39.029073 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-980f952b-191c-4b02-9855-585f14f3ac2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:54:39.156563 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:54:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 22 23:54:39.442341 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee60593-d4fc-4c8f-bb79-2ee03bcc883c None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:39.511432 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: INFO dbcounter [None req-8ee60593-d4fc-4c8f-bb79-2ee03bcc883c None None] Registered counter for database keystone Oct 22 23:54:39.533565 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG oslo_db.sqlalchemy.engines [None req-8ee60593-d4fc-4c8f-bb79-2ee03bcc883c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81381) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 22 23:54:39.539654 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writer thread running {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 22 23:54:39.657877 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee60593-d4fc-4c8f-bb79-2ee03bcc883c None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:39.662162 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323956e0-888e-433e-8ece-3be7d24ab062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:39.662791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323956e0-888e-433e-8ece-3be7d24ab062 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:39.663121 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323956e0-888e-433e-8ece-3be7d24ab062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:39.663449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323956e0-888e-433e-8ece-3be7d24ab062 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:39.672160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323956e0-888e-433e-8ece-3be7d24ab062 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:39.724118 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: WARNING keystone.common.rbac_enforcer.enforcer [None req-323956e0-888e-433e-8ece-3be7d24ab062 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 22 23:54:39.724687 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323956e0-888e-433e-8ece-3be7d24ab062 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:39.733749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Sat Oct 22 23:54:39 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 22 23:54:39.740305 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f952b-191c-4b02-9855-585f14f3ac2f None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:39.757426 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f952b-191c-4b02-9855-585f14f3ac2f None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:39.761811 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65105a3d-28cf-4ed1-8695-f8e55729a81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:39.762902 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65105a3d-28cf-4ed1-8695-f8e55729a81a admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:39.763187 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65105a3d-28cf-4ed1-8695-f8e55729a81a admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:39.763462 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65105a3d-28cf-4ed1-8695-f8e55729a81a admin admin] PATH_INFO: `/v3/projects/e646fb7093394a37b996a9f2520c5910` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:39.765056 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65105a3d-28cf-4ed1-8695-f8e55729a81a admin admin] RBAC: Authorizing `identity:get_project(project_id=e646fb7093394a37b996a9f2520c5910)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:39.820110 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: WARNING keystone.common.rbac_enforcer.enforcer [None req-65105a3d-28cf-4ed1-8695-f8e55729a81a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 22 23:54:39.820450 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65105a3d-28cf-4ed1-8695-f8e55729a81a admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:39.824969 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Sat Oct 22 23:54:39 2022] GET /identity/v3/projects/e646fb7093394a37b996a9f2520c5910 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:54:41.946120 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b705e5-eb6f-456e-9dd3-1c3702a21979 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:41.946640 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b705e5-eb6f-456e-9dd3-1c3702a21979 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:41.946723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b705e5-eb6f-456e-9dd3-1c3702a21979 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:41.950818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:54:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 22 23:54:41.957291 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9986d-a6ff-4085-9bff-3a740674bd34 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:41.957706 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9986d-a6ff-4085-9bff-3a740674bd34 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:41.957851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9986d-a6ff-4085-9bff-3a740674bd34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:41.978432 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-4eb9986d-a6ff-4085-9bff-3a740674bd34 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:54:41.979924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-4eb9986d-a6ff-4085-9bff-3a740674bd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:54:41.987704 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:54:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 22 23:54:42.258212 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b705e5-eb6f-456e-9dd3-1c3702a21979 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:42.270452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b705e5-eb6f-456e-9dd3-1c3702a21979 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:42.274689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997667d7-1d42-4265-8a81-75c6b76aa4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:42.275286 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997667d7-1d42-4265-8a81-75c6b76aa4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:42.275572 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997667d7-1d42-4265-8a81-75c6b76aa4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:42.275851 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997667d7-1d42-4265-8a81-75c6b76aa4b5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:42.284388 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997667d7-1d42-4265-8a81-75c6b76aa4b5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:42.286187 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997667d7-1d42-4265-8a81-75c6b76aa4b5 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:42.294801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Sat Oct 22 23:54:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 22 23:54:42.300689 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9986d-a6ff-4085-9bff-3a740674bd34 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:42.309679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9986d-a6ff-4085-9bff-3a740674bd34 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:42.315259 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c4254-f9cd-4fdf-925d-adc5a18d8c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:42.315885 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c4254-f9cd-4fdf-925d-adc5a18d8c65 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:42.316170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c4254-f9cd-4fdf-925d-adc5a18d8c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:42.316450 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c4254-f9cd-4fdf-925d-adc5a18d8c65 admin admin] PATH_INFO: `/v3/users/ac8d80b2c4a24dfea25c6de809bc37e3` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:42.317795 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c4254-f9cd-4fdf-925d-adc5a18d8c65 admin admin] RBAC: Authorizing `identity:get_user(user_id=ac8d80b2c4a24dfea25c6de809bc37e3)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:42.319615 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c4254-f9cd-4fdf-925d-adc5a18d8c65 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:42.321168 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Sat Oct 22 23:54:42 2022] GET /identity/v3/users/ac8d80b2c4a24dfea25c6de809bc37e3 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:54:44.512294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1b66b-b3bf-4d38-aa74-7f21d1188c00 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:44.512865 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1b66b-b3bf-4d38-aa74-7f21d1188c00 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:44.512865 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1b66b-b3bf-4d38-aa74-7f21d1188c00 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:44.514110 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:54:44.519328 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c52a4-31ff-4860-a958-4f9e5d8a0135 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:44.519720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c52a4-31ff-4860-a958-4f9e5d8a0135 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:44.520011 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c52a4-31ff-4860-a958-4f9e5d8a0135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:44.538215 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-7a3c52a4-31ff-4860-a958-4f9e5d8a0135 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:54:44.539392 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-7a3c52a4-31ff-4860-a958-4f9e5d8a0135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:54:44.575244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0139b2b-e895-4c34-b3e9-5613d89d6e77 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:44.575530 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0139b2b-e895-4c34-b3e9-5613d89d6e77 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:44.575807 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0139b2b-e895-4c34-b3e9-5613d89d6e77 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:44.577157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 6/11] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 22 23:54:44.582222 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79340789-412f-4049-9a3d-ace30d151982 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:44.582505 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79340789-412f-4049-9a3d-ace30d151982 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:44.582807 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79340789-412f-4049-9a3d-ace30d151982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:44.621042 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 6/12] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:54:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:54:44.625769 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG passlib.handlers.bcrypt [None req-79340789-412f-4049-9a3d-ace30d151982 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81381) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 22 23:54:44.626195 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG passlib.handlers.bcrypt [None req-79340789-412f-4049-9a3d-ace30d151982 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81381) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 22 23:54:44.675472 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-79340789-412f-4049-9a3d-ace30d151982 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:54:44.677063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-79340789-412f-4049-9a3d-ace30d151982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:54:44.775755 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 7/13] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:54:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 22 23:54:44.888758 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c52a4-31ff-4860-a958-4f9e5d8a0135 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:44.914472 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:54:44.994129 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c52a4-31ff-4860-a958-4f9e5d8a0135 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:45.026595 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1bbad-8947-4492-a49c-580452a6bc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:45.027168 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1bbad-8947-4492-a49c-580452a6bc76 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:45.027453 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1bbad-8947-4492-a49c-580452a6bc76 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:45.027731 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1bbad-8947-4492-a49c-580452a6bc76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:45.038982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc1bbad-8947-4492-a49c-580452a6bc76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:54:45.044993 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79340789-412f-4049-9a3d-ace30d151982 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:45.047924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-edc1bbad-8947-4492-a49c-580452a6bc76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:54:45.047924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:54:45.047924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:54:45.047924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:54:45.047924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:54:45.047924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:54:45.047924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:54:45.047924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:54:45.047924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:54:45.047924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:54:45.047924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:54:45.047924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:54:45.047924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:54:45.047924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:54:45.047924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:54:45.047924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:54:45.049051 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:54:45.049051 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:54:45.049051 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:54:45.049051 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:54:45.049051 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:54:45.049051 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:54:45.049051 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:54:45.049051 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:54:45.049051 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:54:45.049051 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:54:45.049051 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:54:45.049051 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:54:45.049051 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:54:45.049051 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:54:45.049051 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:54:45.049051 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:54:45.050347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:54:45.050347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:54:45.050347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:54:45.050347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:54:45.050347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:54:45.050347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:54:45.050347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:54:45.050347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:54:45.050347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:54:45.050347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:54:45.050347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:54:45.050347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:54:45.050347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 7/14] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Oct 22 23:54:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:54:45.054637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1bbad-8947-4492-a49c-580452a6bc76 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:45.138314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:54:45.150778 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79340789-412f-4049-9a3d-ace30d151982 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:45.153067 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:54:45.158732 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1bbad-8947-4492-a49c-580452a6bc76 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:45.190972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0706c92d-92a8-4cab-8e4e-b18e8270bd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:45.191835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706c92d-92a8-4cab-8e4e-b18e8270bd44 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:45.192118 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706c92d-92a8-4cab-8e4e-b18e8270bd44 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:45.192395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706c92d-92a8-4cab-8e4e-b18e8270bd44 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:45.193386 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0706c92d-92a8-4cab-8e4e-b18e8270bd44 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:45.193687 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085fe418-f46b-4173-9f90-a40b5dbc7f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:45.194297 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fe418-f46b-4173-9f90-a40b5dbc7f2d None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:45.194581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fe418-f46b-4173-9f90-a40b5dbc7f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:45.194796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0706c92d-92a8-4cab-8e4e-b18e8270bd44 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:45.194888 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fe418-f46b-4173-9f90-a40b5dbc7f2d None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:45.195519 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fe418-f46b-4173-9f90-a40b5dbc7f2d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:54:45.195825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fe418-f46b-4173-9f90-a40b5dbc7f2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:45.197234 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fe418-f46b-4173-9f90-a40b5dbc7f2d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:45.206969 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.sql.core [None req-0706c92d-92a8-4cab-8e4e-b18e8270bd44 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 22 23:54:45.206969 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 22 23:54:45.206969 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [parameters: {'id': '6d3d96feffa648d2bb1ac86ce9cd20e0', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 22 23:54:45.206969 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81381) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 22 23:54:45.209354 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 8/15] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:54:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:54:45.215040 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085fe418-f46b-4173-9f90-a40b5dbc7f2d None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:45.225721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-0706c92d-92a8-4cab-8e4e-b18e8270bd44 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 22 23:54:45.225721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:54:45.225721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 22 23:54:45.225721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 22 23:54:45.225721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 22 23:54:45.225721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 22 23:54:45.225721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 22 23:54:45.225721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result = self._query(query) Oct 22 23:54:45.225721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 22 23:54:45.225721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application conn.query(q) Oct 22 23:54:45.225721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 22 23:54:45.225721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 22 23:54:45.225721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 22 23:54:45.225721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result.read() Oct 22 23:54:45.226723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 22 23:54:45.226723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 22 23:54:45.226723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 22 23:54:45.226723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 22 23:54:45.226723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 22 23:54:45.226723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 22 23:54:45.226723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 22 23:54:45.226723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 22 23:54:45.226723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 22 23:54:45.226723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:54:45.226723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 22 23:54:45.226723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:54:45.226723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:54:45.226723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 22 23:54:45.226723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 22 23:54:45.226723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 22 23:54:45.226723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 22 23:54:45.226723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 22 23:54:45.228180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application next(self.gen) Oct 22 23:54:45.228180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 22 23:54:45.228180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application yield resource Oct 22 23:54:45.228180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 22 23:54:45.228180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application next(self.gen) Oct 22 23:54:45.228180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 22 23:54:45.228180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.session.rollback() Oct 22 23:54:45.228180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 22 23:54:45.228180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.force_reraise() Oct 22 23:54:45.228180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 22 23:54:45.228180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise self.value Oct 22 23:54:45.228180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 22 23:54:45.228180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 22 23:54:45.228180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 22 23:54:45.228180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application session.commit() Oct 22 23:54:45.228180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 22 23:54:45.228180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 22 23:54:45.229309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 22 23:54:45.229309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._prepare_impl() Oct 22 23:54:45.229309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 22 23:54:45.229309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.session.flush() Oct 22 23:54:45.229309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 22 23:54:45.229309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._flush(objects) Oct 22 23:54:45.229309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 22 23:54:45.229309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 22 23:54:45.229309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 22 23:54:45.229309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application compat.raise_( Oct 22 23:54:45.229309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 22 23:54:45.229309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception Oct 22 23:54:45.229309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 22 23:54:45.229309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application flush_context.execute() Oct 22 23:54:45.229309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 22 23:54:45.229309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rec.execute(self) Oct 22 23:54:45.229309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 22 23:54:45.230361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 22 23:54:45.230361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 22 23:54:45.230361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 22 23:54:45.230361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 22 23:54:45.230361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 22 23:54:45.230361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 22 23:54:45.230361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 22 23:54:45.230361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 22 23:54:45.230361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 22 23:54:45.230361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 22 23:54:45.230361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 22 23:54:45.230361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 22 23:54:45.230361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 22 23:54:45.230361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 22 23:54:45.230361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 22 23:54:45.231335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 22 23:54:45.231335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception Oct 22 23:54:45.231335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 22 23:54:45.231335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 22 23:54:45.231335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 22 23:54:45.231335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 22 23:54:45.231335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 22 23:54:45.231335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result = self._query(query) Oct 22 23:54:45.231335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 22 23:54:45.231335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application conn.query(q) Oct 22 23:54:45.231335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 22 23:54:45.231335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 22 23:54:45.231335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 22 23:54:45.231335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result.read() Oct 22 23:54:45.231335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 22 23:54:45.231335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 22 23:54:45.231335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 22 23:54:45.232691 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 22 23:54:45.232691 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 22 23:54:45.232691 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 22 23:54:45.232691 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 22 23:54:45.232691 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 22 23:54:45.232691 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 22 23:54:45.232691 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 22 23:54:45.232691 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [parameters: {'id': '6d3d96feffa648d2bb1ac86ce9cd20e0', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 22 23:54:45.232691 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 22 23:54:45.232691 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:54:45.232691 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:54:45.232691 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:54:45.232691 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:54:45.232691 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 22 23:54:45.232691 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 22 23:54:45.233948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 22 23:54:45.233948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 22 23:54:45.233948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 22 23:54:45.233948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:54:45.233948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:54:45.233948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:54:45.233948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:54:45.233948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:54:45.233948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:54:45.233948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:54:45.233948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:54:45.233948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:54:45.233948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:54:45.233948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:54:45.233948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:54:45.233948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:54:45.233948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:54:45.235385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 22 23:54:45.235385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 22 23:54:45.235385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:54:45.235385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:54:45.235385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 22 23:54:45.235385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 22 23:54:45.235385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:54:45.235385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:54:45.235385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 22 23:54:45.235385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 22 23:54:45.235385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 22 23:54:45.235385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:54:45.235385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 8/16] 10.222.0.13 () {60 vars in 1137 bytes} [Sat Oct 22 23:54:45 2022] POST /identity/v3/domains => generated 190 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:54:45.235385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0706c92d-92a8-4cab-8e4e-b18e8270bd44 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:45.316714 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085fe418-f46b-4173-9f90-a40b5dbc7f2d None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:45.334069 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0706c92d-92a8-4cab-8e4e-b18e8270bd44 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:45.351100 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3027f-64cd-48ea-98f9-acac79cea561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:45.351740 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3027f-64cd-48ea-98f9-acac79cea561 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:45.352021 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3027f-64cd-48ea-98f9-acac79cea561 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:45.352295 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3027f-64cd-48ea-98f9-acac79cea561 None admin] PATH_INFO: `/v3/users/ac8d80b2c4a24dfea25c6de809bc37e3` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:45.353466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3027f-64cd-48ea-98f9-acac79cea561 None admin] RBAC: Authorizing `identity:get_user(user_id=ac8d80b2c4a24dfea25c6de809bc37e3)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:45.354916 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3027f-64cd-48ea-98f9-acac79cea561 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:45.356466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 9/17] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:54:45 2022] GET /identity/v3/users/ac8d80b2c4a24dfea25c6de809bc37e3 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:54:45.362172 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3027f-64cd-48ea-98f9-acac79cea561 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:45.367894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f263d-5174-42fe-b5f4-40b03a38d794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:45.368781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f263d-5174-42fe-b5f4-40b03a38d794 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:45.369067 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f263d-5174-42fe-b5f4-40b03a38d794 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:45.369341 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f263d-5174-42fe-b5f4-40b03a38d794 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:45.377276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:54:45.388591 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f263d-5174-42fe-b5f4-40b03a38d794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:45.390054 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f263d-5174-42fe-b5f4-40b03a38d794 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:45.410001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-ec7f263d-5174-42fe-b5f4-40b03a38d794 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:54:45.410001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:54:45.410001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:54:45.410001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:54:45.410001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:54:45.410001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:54:45.410001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:54:45.410001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:54:45.410001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:54:45.410001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:54:45.410001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:54:45.410001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:54:45.410001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:54:45.410001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:54:45.410001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:54:45.410001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:54:45.411084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:54:45.411084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:54:45.411084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:54:45.411084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:54:45.411084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:54:45.411084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:54:45.411084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:54:45.411084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:54:45.411084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:54:45.411084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:54:45.411084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:54:45.411084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:54:45.411084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:54:45.411084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:54:45.411084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:54:45.411084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:54:45.411084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:54:45.411888 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:54:45.411888 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:54:45.411888 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:54:45.411888 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 9/18] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:54:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:54:45.416874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f263d-5174-42fe-b5f4-40b03a38d794 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:45.440006 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:54:45.464103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3027f-64cd-48ea-98f9-acac79cea561 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:45.497490 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6aae6-97da-4dd4-9072-9da5f36423e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:45.498115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6aae6-97da-4dd4-9072-9da5f36423e7 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:45.498397 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6aae6-97da-4dd4-9072-9da5f36423e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:45.498672 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6aae6-97da-4dd4-9072-9da5f36423e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:45.499809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf6aae6-97da-4dd4-9072-9da5f36423e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:45.501335 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf6aae6-97da-4dd4-9072-9da5f36423e7 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:45.502887 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 10/19] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:54:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:54:45.508383 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6aae6-97da-4dd4-9072-9da5f36423e7 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:45.519677 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f263d-5174-42fe-b5f4-40b03a38d794 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:45.553010 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ce32a-68e9-49dd-a3ed-3f2693152e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:45.553615 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ce32a-68e9-49dd-a3ed-3f2693152e1e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:45.553905 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ce32a-68e9-49dd-a3ed-3f2693152e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:45.554181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ce32a-68e9-49dd-a3ed-3f2693152e1e None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:45.554817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ce32a-68e9-49dd-a3ed-3f2693152e1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:54:45.555127 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ce32a-68e9-49dd-a3ed-3f2693152e1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:45.556584 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ce32a-68e9-49dd-a3ed-3f2693152e1e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:45.574355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 10/20] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:54:45.587009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:54:45.608871 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6aae6-97da-4dd4-9072-9da5f36423e7 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:45.645454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd7470c-ecf5-4cb1-9a35-147eb69debb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:45.646141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7470c-ecf5-4cb1-9a35-147eb69debb9 None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:45.646424 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7470c-ecf5-4cb1-9a35-147eb69debb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:45.646702 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7470c-ecf5-4cb1-9a35-147eb69debb9 None admin] PATH_INFO: `/v3/domains/default/users/ac8d80b2c4a24dfea25c6de809bc37e3/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:45.648723 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd7470c-ecf5-4cb1-9a35-147eb69debb9 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ac8d80b2c4a24dfea25c6de809bc37e3, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:45.650183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd7470c-ecf5-4cb1-9a35-147eb69debb9 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:45.766595 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 11/21] 10.222.0.13 () {58 vars in 1361 bytes} [Sat Oct 22 23:54:45 2022] PUT /identity/v3/domains/default/users/ac8d80b2c4a24dfea25c6de809bc37e3/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 22 23:54:48.169279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e3107-d937-4116-b2e8-58fbb77e3322 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:48.170246 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e3107-d937-4116-b2e8-58fbb77e3322 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:48.170723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e3107-d937-4116-b2e8-58fbb77e3322 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:48.172759 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 11/22] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:54:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:54:48.178609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b6b93-0fe6-49a1-a04c-2a0da5e01651 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:48.179512 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b6b93-0fe6-49a1-a04c-2a0da5e01651 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:48.180020 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b6b93-0fe6-49a1-a04c-2a0da5e01651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:48.205842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-349b6b93-0fe6-49a1-a04c-2a0da5e01651 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:54:48.207745 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-349b6b93-0fe6-49a1-a04c-2a0da5e01651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:54:48.289095 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 12/23] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:54:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:54:48.367891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee783ed9-1008-4df6-bc59-d1069a58f8fb None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:48.368501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee783ed9-1008-4df6-bc59-d1069a58f8fb None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:48.368981 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee783ed9-1008-4df6-bc59-d1069a58f8fb None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:48.370735 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 12/24] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:54:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:54:48.377033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e7830-fca9-412f-a44f-714692e92c5e None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:48.377579 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e7830-fca9-412f-a44f-714692e92c5e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:48.378094 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e7830-fca9-412f-a44f-714692e92c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:48.405570 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-615e7830-fca9-412f-a44f-714692e92c5e None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:54:48.407242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-615e7830-fca9-412f-a44f-714692e92c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:54:48.487959 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:54:48.493473 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 13/25] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:54:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 22 23:54:48.701949 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20646f-492a-4f58-8253-8b0d62db549a None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:48.702549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20646f-492a-4f58-8253-8b0d62db549a None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:48.703553 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b20646f-492a-4f58-8253-8b0d62db549a None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:48.705360 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 13/26] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:54:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:54:48.714348 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58069628-37c0-4924-8ce5-4c8f33da0b7a None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:48.714880 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58069628-37c0-4924-8ce5-4c8f33da0b7a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:48.715357 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58069628-37c0-4924-8ce5-4c8f33da0b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:48.741086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-58069628-37c0-4924-8ce5-4c8f33da0b7a None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:54:48.742794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef756a-bc3a-4227-9450-75a4093ba5be None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:48.743520 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef756a-bc3a-4227-9450-75a4093ba5be None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:48.744000 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef756a-bc3a-4227-9450-75a4093ba5be None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:48.745819 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 14/27] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:54:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:54:48.745988 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-58069628-37c0-4924-8ce5-4c8f33da0b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:54:48.746873 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef756a-bc3a-4227-9450-75a4093ba5be None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:48.851599 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 14/28] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:54:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:54:48.853142 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c1f36-9dbf-4966-85e0-8c5de569f7ec None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:48.853436 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c1f36-9dbf-4966-85e0-8c5de569f7ec None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:48.853721 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c1f36-9dbf-4966-85e0-8c5de569f7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:48.888564 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-8b3c1f36-9dbf-4966-85e0-8c5de569f7ec None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:54:48.889776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8b3c1f36-9dbf-4966-85e0-8c5de569f7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:54:48.899390 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:54:48.932363 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef756a-bc3a-4227-9450-75a4093ba5be None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:49.004446 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31976101-86d2-4f61-9dcf-f6787a8ba368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:49.006510 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31976101-86d2-4f61-9dcf-f6787a8ba368 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:49.007244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31976101-86d2-4f61-9dcf-f6787a8ba368 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:49.008712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31976101-86d2-4f61-9dcf-f6787a8ba368 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:49.009608 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31976101-86d2-4f61-9dcf-f6787a8ba368 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:49.012067 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31976101-86d2-4f61-9dcf-f6787a8ba368 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:49.025816 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 15/29] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:54:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:54:49.027627 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c1f36-9dbf-4966-85e0-8c5de569f7ec None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:49.111066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:54:49.154173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c1f36-9dbf-4966-85e0-8c5de569f7ec None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:49.209216 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5bba7-f17e-485a-b5b2-a02fcab75836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:49.210681 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5bba7-f17e-485a-b5b2-a02fcab75836 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:49.211192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5bba7-f17e-485a-b5b2-a02fcab75836 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:49.211670 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5bba7-f17e-485a-b5b2-a02fcab75836 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:49.216354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 15/30] 10.222.0.13 () {60 vars in 1131 bytes} [Sat Oct 22 23:54:48 2022] POST /identity/v3/roles => generated 219 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:54:49.235375 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5bba7-f17e-485a-b5b2-a02fcab75836 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:54:49.248779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-3af5bba7-f17e-485a-b5b2-a02fcab75836 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:54:49.248779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:54:49.248779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:54:49.248779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:54:49.248779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:54:49.248779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:54:49.248779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:54:49.248779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:54:49.248779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:54:49.248779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:54:49.248779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:54:49.248779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:54:49.248779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:54:49.248779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:54:49.248779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:54:49.248779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:54:49.249741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:54:49.249741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:54:49.249741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:54:49.249741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:54:49.249741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:54:49.249741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:54:49.249741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:54:49.249741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:54:49.249741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:54:49.249741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:54:49.249741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:54:49.249741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:54:49.249741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:54:49.249741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:54:49.249741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:54:49.249741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:54:49.250761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:54:49.250761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:54:49.250761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:54:49.250761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:54:49.250761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:54:49.250761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:54:49.250761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:54:49.250761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:54:49.250761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:54:49.250761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:54:49.250761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:54:49.250761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:54:49.250761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 16/31] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Oct 22 23:54:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:54:49.256328 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31976101-86d2-4f61-9dcf-f6787a8ba368 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:49.340685 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5bba7-f17e-485a-b5b2-a02fcab75836 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:49.387345 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31976101-86d2-4f61-9dcf-f6787a8ba368 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:49.427168 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb2191-85e7-469f-8a78-4cac6ef8236a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:49.428138 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb2191-85e7-469f-8a78-4cac6ef8236a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:49.428665 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb2191-85e7-469f-8a78-4cac6ef8236a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:49.429169 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb2191-85e7-469f-8a78-4cac6ef8236a None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:49.430094 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb2191-85e7-469f-8a78-4cac6ef8236a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:54:49.430603 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb2191-85e7-469f-8a78-4cac6ef8236a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:49.432602 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb2191-85e7-469f-8a78-4cac6ef8236a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:49.457502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 16/32] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:54:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:54:49.462751 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb2191-85e7-469f-8a78-4cac6ef8236a None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:49.467157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:54:49.544681 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5bba7-f17e-485a-b5b2-a02fcab75836 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:49.573682 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:54:49.591740 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2faaf-d4dd-446c-9db8-3c341363a1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:49.592356 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2faaf-d4dd-446c-9db8-3c341363a1c2 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:49.592673 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2faaf-d4dd-446c-9db8-3c341363a1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:49.592960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2faaf-d4dd-446c-9db8-3c341363a1c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:49.594393 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a2faaf-d4dd-446c-9db8-3c341363a1c2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:49.596264 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a2faaf-d4dd-446c-9db8-3c341363a1c2 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:49.623634 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 17/33] 10.222.0.13 () {60 vars in 1131 bytes} [Sat Oct 22 23:54:49 2022] POST /identity/v3/roles => generated 225 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 22 23:54:49.624584 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2faaf-d4dd-446c-9db8-3c341363a1c2 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:49.628206 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb2191-85e7-469f-8a78-4cac6ef8236a None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:49.691752 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606b948-a770-422f-b09a-21a3ded502d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:49.692824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606b948-a770-422f-b09a-21a3ded502d3 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:49.693333 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606b948-a770-422f-b09a-21a3ded502d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:49.693827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606b948-a770-422f-b09a-21a3ded502d3 None admin] PATH_INFO: `/v3/users/ac8d80b2c4a24dfea25c6de809bc37e3` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:49.695820 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e606b948-a770-422f-b09a-21a3ded502d3 None admin] RBAC: Authorizing `identity:get_user(user_id=ac8d80b2c4a24dfea25c6de809bc37e3)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:49.697581 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e606b948-a770-422f-b09a-21a3ded502d3 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:49.700215 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 17/34] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:54:49 2022] GET /identity/v3/users/ac8d80b2c4a24dfea25c6de809bc37e3 => generated 247 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:54:49.702231 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe54f13-f4ce-4d40-b6c5-315cc578a41f None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:49.702798 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe54f13-f4ce-4d40-b6c5-315cc578a41f None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:49.703536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe54f13-f4ce-4d40-b6c5-315cc578a41f None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:49.705796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 18/35] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:54:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:54:49.709022 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe54f13-f4ce-4d40-b6c5-315cc578a41f None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:49.750832 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2faaf-d4dd-446c-9db8-3c341363a1c2 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:49.804445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:54:49.847335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe54f13-f4ce-4d40-b6c5-315cc578a41f None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:49.888252 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd55338-41e7-4246-9d87-fa6384f75fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:49.888928 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd55338-41e7-4246-9d87-fa6384f75fbe None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:49.889222 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd55338-41e7-4246-9d87-fa6384f75fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:49.889499 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd55338-41e7-4246-9d87-fa6384f75fbe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:49.892134 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd55338-41e7-4246-9d87-fa6384f75fbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:49.892780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd55338-41e7-4246-9d87-fa6384f75fbe None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:49.894694 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 19/36] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:54:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:54:49.895637 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27154f1-9e64-42f5-b7c8-da85cc8aaece None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:49.895926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27154f1-9e64-42f5-b7c8-da85cc8aaece None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:49.896212 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27154f1-9e64-42f5-b7c8-da85cc8aaece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:49.902511 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb79b5f-0be5-4dbe-a87c-14ba0d90cfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:49.903170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb79b5f-0be5-4dbe-a87c-14ba0d90cfd7 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:49.903456 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb79b5f-0be5-4dbe-a87c-14ba0d90cfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:49.903735 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb79b5f-0be5-4dbe-a87c-14ba0d90cfd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:49.916650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb79b5f-0be5-4dbe-a87c-14ba0d90cfd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:49.917928 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-b27154f1-9e64-42f5-b7c8-da85cc8aaece None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:54:49.918132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb79b5f-0be5-4dbe-a87c-14ba0d90cfd7 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:49.919525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b27154f1-9e64-42f5-b7c8-da85cc8aaece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:54:49.934965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-1eb79b5f-0be5-4dbe-a87c-14ba0d90cfd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:54:49.934965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:54:49.934965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:54:49.934965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:54:49.934965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:54:49.934965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:54:49.934965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:54:49.934965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:54:49.934965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:54:49.934965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:54:49.934965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:54:49.934965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:54:49.934965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:54:49.934965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:54:49.934965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:54:49.934965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:54:49.935870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:54:49.935870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:54:49.935870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:54:49.935870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:54:49.935870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:54:49.935870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:54:49.935870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:54:49.935870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:54:49.935870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:54:49.935870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:54:49.935870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:54:49.935870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:54:49.935870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:54:49.935870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:54:49.935870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:54:49.935870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:54:49.935870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:54:49.937792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:54:49.937792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:54:49.937792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:54:49.937792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 18/37] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:54:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:54:49.937792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb79b5f-0be5-4dbe-a87c-14ba0d90cfd7 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:49.986321 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:54:50.024064 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 20/38] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:54:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:54:50.024970 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27154f1-9e64-42f5-b7c8-da85cc8aaece None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:50.079372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb79b5f-0be5-4dbe-a87c-14ba0d90cfd7 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:50.117669 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:54:50.126174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d7280-d1f2-480f-b329-31e325f04fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:50.126796 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d7280-d1f2-480f-b329-31e325f04fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:50.127086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d7280-d1f2-480f-b329-31e325f04fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:50.127368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d7280-d1f2-480f-b329-31e325f04fb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:50.128112 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d7280-d1f2-480f-b329-31e325f04fb5 None admin] RBAC: Adding query filter params (role.id=839a55263e2c4cbaa2f945b75cf298d6, scope.domain.id=default, user.id=ac8d80b2c4a24dfea25c6de809bc37e3) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:54:50.128424 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d7280-d1f2-480f-b329-31e325f04fb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:50.129907 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0d7280-d1f2-480f-b329-31e325f04fb5 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:50.137321 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 19/39] 10.222.0.13 () {56 vars in 1322 bytes} [Sat Oct 22 23:54:49 2022] GET /identity/v3/role_assignments?user.id=ac8d80b2c4a24dfea25c6de809bc37e3&scope.domain.id=default&role.id=839a55263e2c4cbaa2f945b75cf298d6 => generated 530 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:54:50.162403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27154f1-9e64-42f5-b7c8-da85cc8aaece None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:50.195088 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d6c89-7006-4c27-b59a-85dfc463cbc9 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:50.195429 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d6c89-7006-4c27-b59a-85dfc463cbc9 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:50.195667 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d6c89-7006-4c27-b59a-85dfc463cbc9 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:50.197293 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 20/40] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 22 23:54:50.203458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db888aa-4f2d-41d1-a694-f6a71b93a883 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:50.203757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db888aa-4f2d-41d1-a694-f6a71b93a883 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:50.204042 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db888aa-4f2d-41d1-a694-f6a71b93a883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:50.208868 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e68fe-d48b-4b0f-9289-d9eb88653dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:50.209907 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e68fe-d48b-4b0f-9289-d9eb88653dca None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:50.210449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e68fe-d48b-4b0f-9289-d9eb88653dca None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:50.211228 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e68fe-d48b-4b0f-9289-d9eb88653dca None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:50.213190 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e68fe-d48b-4b0f-9289-d9eb88653dca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:54:50.213507 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e68fe-d48b-4b0f-9289-d9eb88653dca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:50.214935 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e68fe-d48b-4b0f-9289-d9eb88653dca None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:50.227773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 21/41] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:54:50.229843 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-5db888aa-4f2d-41d1-a694-f6a71b93a883 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:54:50.231641 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-5db888aa-4f2d-41d1-a694-f6a71b93a883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:54:50.233713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e68fe-d48b-4b0f-9289-d9eb88653dca None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:50.313819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 21/42] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:54:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 22 23:54:50.339237 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db888aa-4f2d-41d1-a694-f6a71b93a883 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:50.352550 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e68fe-d48b-4b0f-9289-d9eb88653dca None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:50.376397 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:54:50.388291 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:54:50.403850 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d744f-c085-4909-a155-5e9e0a7eb450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:50.407785 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d744f-c085-4909-a155-5e9e0a7eb450 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:50.407785 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d744f-c085-4909-a155-5e9e0a7eb450 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:50.407785 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d744f-c085-4909-a155-5e9e0a7eb450 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:50.407785 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d744f-c085-4909-a155-5e9e0a7eb450 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:50.408093 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d744f-c085-4909-a155-5e9e0a7eb450 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:50.444419 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 22/43] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Oct 22 23:54:50 2022] POST /identity/v3/projects => generated 301 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:54:50.461853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db888aa-4f2d-41d1-a694-f6a71b93a883 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:50.502326 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd8452-7e95-49b8-abe0-e144a676080c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:50.502970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd8452-7e95-49b8-abe0-e144a676080c None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:50.503257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd8452-7e95-49b8-abe0-e144a676080c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:50.503538 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd8452-7e95-49b8-abe0-e144a676080c None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:50.504232 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd8452-7e95-49b8-abe0-e144a676080c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:50.505715 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd8452-7e95-49b8-abe0-e144a676080c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:50.524545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 22/44] 10.222.0.13 () {60 vars in 1131 bytes} [Sat Oct 22 23:54:50 2022] POST /identity/v3/roles => generated 223 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 22 23:54:50.588093 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d744f-c085-4909-a155-5e9e0a7eb450 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:50.706613 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d744f-c085-4909-a155-5e9e0a7eb450 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:50.741275 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc985ec-a004-49e4-9f5e-a60b2509e9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:50.741912 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc985ec-a004-49e4-9f5e-a60b2509e9e8 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:50.742198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc985ec-a004-49e4-9f5e-a60b2509e9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:50.742481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc985ec-a004-49e4-9f5e-a60b2509e9e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:50.743577 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc985ec-a004-49e4-9f5e-a60b2509e9e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:50.745059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc985ec-a004-49e4-9f5e-a60b2509e9e8 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:50.746801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 23/45] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:54:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:54:50.752556 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd8452-7e95-49b8-abe0-e144a676080c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:50.836686 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:54:50.852310 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd8452-7e95-49b8-abe0-e144a676080c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:50.885500 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745718d-46b0-4388-bf76-5dc1f5f8c3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:50.886101 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745718d-46b0-4388-bf76-5dc1f5f8c3cf None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:50.886386 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745718d-46b0-4388-bf76-5dc1f5f8c3cf None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:50.886661 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745718d-46b0-4388-bf76-5dc1f5f8c3cf None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:50.887309 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745718d-46b0-4388-bf76-5dc1f5f8c3cf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:50.888780 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745718d-46b0-4388-bf76-5dc1f5f8c3cf None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:50.915639 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 23/46] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Oct 22 23:54:50 2022] POST /identity/v3/projects => generated 312 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:54:53.055806 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e2d9e-ac41-43fb-badd-2cf8092121e1 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:53.056325 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e2d9e-ac41-43fb-badd-2cf8092121e1 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:53.056433 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e2d9e-ac41-43fb-badd-2cf8092121e1 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:53.057764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 24/47] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:54:53.063907 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57503fc-03ab-468d-bd63-9c971450310c None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:53.064594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57503fc-03ab-468d-bd63-9c971450310c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:53.064782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57503fc-03ab-468d-bd63-9c971450310c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:53.082983 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-b57503fc-03ab-468d-bd63-9c971450310c None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:54:53.084163 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b57503fc-03ab-468d-bd63-9c971450310c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:54:53.149955 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 24/48] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:54:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:54:53.418281 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e2d9e-ac41-43fb-badd-2cf8092121e1 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:53.468409 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:54:53.521308 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e2d9e-ac41-43fb-badd-2cf8092121e1 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:53.554396 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d67d61-1b42-44ab-a8e3-4370f29a1ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:53.555010 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d67d61-1b42-44ab-a8e3-4370f29a1ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:53.555293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d67d61-1b42-44ab-a8e3-4370f29a1ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:53.555568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d67d61-1b42-44ab-a8e3-4370f29a1ac5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:53.556688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d67d61-1b42-44ab-a8e3-4370f29a1ac5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:53.558119 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d67d61-1b42-44ab-a8e3-4370f29a1ac5 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:53.559648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 25/49] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:54:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:54:53.565952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57503fc-03ab-468d-bd63-9c971450310c None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:53.629509 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:54:53.662728 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57503fc-03ab-468d-bd63-9c971450310c None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:53.696604 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494c645-f312-4926-b77f-971714f7e595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:53.697134 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494c645-f312-4926-b77f-971714f7e595 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:53.697419 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494c645-f312-4926-b77f-971714f7e595 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:53.697696 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494c645-f312-4926-b77f-971714f7e595 None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:53.698337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e494c645-f312-4926-b77f-971714f7e595 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:53.699791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e494c645-f312-4926-b77f-971714f7e595 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:53.724316 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 25/50] 10.222.0.13 () {60 vars in 1140 bytes} [Sat Oct 22 23:54:53 2022] POST /identity/v3/projects => generated 298 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:54:55.868705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9f761-3a2f-4bfb-bbd5-cf3bc94ccf24 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:55.869219 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9f761-3a2f-4bfb-bbd5-cf3bc94ccf24 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:55.869324 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9f761-3a2f-4bfb-bbd5-cf3bc94ccf24 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:55.870599 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 26/51] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:54:55.876355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d9a99-e0d3-4229-aabd-657cb7ad0b00 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:55.876783 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d9a99-e0d3-4229-aabd-657cb7ad0b00 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:55.876931 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d9a99-e0d3-4229-aabd-657cb7ad0b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:55.896113 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-ea8d9a99-e0d3-4229-aabd-657cb7ad0b00 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:54:55.897410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ea8d9a99-e0d3-4229-aabd-657cb7ad0b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:54:55.962777 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 26/52] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:54:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:54:56.233547 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9f761-3a2f-4bfb-bbd5-cf3bc94ccf24 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:56.438273 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9f761-3a2f-4bfb-bbd5-cf3bc94ccf24 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:56.454785 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:54:56.472120 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b4773-6f8d-43ef-9e98-611dbf56f520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:56.472791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b4773-6f8d-43ef-9e98-611dbf56f520 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:56.473082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b4773-6f8d-43ef-9e98-611dbf56f520 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:56.473364 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b4773-6f8d-43ef-9e98-611dbf56f520 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:56.474465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b4773-6f8d-43ef-9e98-611dbf56f520 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:56.475888 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16b4773-6f8d-43ef-9e98-611dbf56f520 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:56.477481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 27/53] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:54:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:54:56.483188 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d9a99-e0d3-4229-aabd-657cb7ad0b00 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:54:56.531606 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:54:56.581767 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d9a99-e0d3-4229-aabd-657cb7ad0b00 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:54:56.614129 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a0fbb-b23f-47d3-9df9-59572e506f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:54:56.614732 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a0fbb-b23f-47d3-9df9-59572e506f83 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:56.615014 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a0fbb-b23f-47d3-9df9-59572e506f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:56.615289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a0fbb-b23f-47d3-9df9-59572e506f83 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:56.615947 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a0fbb-b23f-47d3-9df9-59572e506f83 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:54:56.617427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a0fbb-b23f-47d3-9df9-59572e506f83 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:54:56.643821 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 27/54] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Oct 22 23:54:56 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:54:59.908996 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486aea0-03f5-4d56-8f1b-c5173c9cac10 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:59.910060 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486aea0-03f5-4d56-8f1b-c5173c9cac10 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:59.910549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486aea0-03f5-4d56-8f1b-c5173c9cac10 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:59.912092 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 28/55] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:54:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:54:59.917795 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4322ac-de5c-428d-a087-ada1eca86f72 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:54:59.918611 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4322ac-de5c-428d-a087-ada1eca86f72 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:54:59.919139 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4322ac-de5c-428d-a087-ada1eca86f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:54:59.949689 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-0d4322ac-de5c-428d-a087-ada1eca86f72 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:54:59.951439 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-0d4322ac-de5c-428d-a087-ada1eca86f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:00.047416 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 28/56] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:54:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:00.145076 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a35f94-52e7-4b6c-a1ae-fe5242e456c1 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:00.145710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a35f94-52e7-4b6c-a1ae-fe5242e456c1 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:00.146219 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a35f94-52e7-4b6c-a1ae-fe5242e456c1 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:00.147761 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 29/57] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:00.153048 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c33813-1533-4a06-99bc-2415e8c79803 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:00.153342 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c33813-1533-4a06-99bc-2415e8c79803 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:00.153630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c33813-1533-4a06-99bc-2415e8c79803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:00.182127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-55c33813-1533-4a06-99bc-2415e8c79803 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:00.183270 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-55c33813-1533-4a06-99bc-2415e8c79803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:00.209254 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68a2fe-ff9a-4351-bb62-a155f60cbd21 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:00.209893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68a2fe-ff9a-4351-bb62-a155f60cbd21 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:00.210376 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68a2fe-ff9a-4351-bb62-a155f60cbd21 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:00.211923 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 30/58] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:00.217273 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa80e4-90ab-4d2e-96be-cc21284ada76 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:00.217801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa80e4-90ab-4d2e-96be-cc21284ada76 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:00.218333 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa80e4-90ab-4d2e-96be-cc21284ada76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:00.228828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:00.253370 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-7ffa80e4-90ab-4d2e-96be-cc21284ada76 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:00.254588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-7ffa80e4-90ab-4d2e-96be-cc21284ada76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:00.301852 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 29/59] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:00.369928 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 31/60] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:00.409861 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61571435-c84c-4bf3-8e66-2237137115d4 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:00.410472 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61571435-c84c-4bf3-8e66-2237137115d4 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:00.410971 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61571435-c84c-4bf3-8e66-2237137115d4 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:00.412556 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 30/61] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:00.417956 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45667f3-44c3-42eb-ba14-16e2867eb994 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:00.418513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45667f3-44c3-42eb-ba14-16e2867eb994 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:00.418993 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45667f3-44c3-42eb-ba14-16e2867eb994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:00.438089 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-b45667f3-44c3-42eb-ba14-16e2867eb994 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:00.439878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b45667f3-44c3-42eb-ba14-16e2867eb994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:00.451520 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61571435-c84c-4bf3-8e66-2237137115d4 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:00.510531 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 32/62] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:00.604392 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45667f3-44c3-42eb-ba14-16e2867eb994 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:00.604637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61571435-c84c-4bf3-8e66-2237137115d4 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:00.642968 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a7b64b-0528-4c5c-a016-b3a4441e0484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:00.643612 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7b64b-0528-4c5c-a016-b3a4441e0484 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:00.643947 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7b64b-0528-4c5c-a016-b3a4441e0484 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:00.644338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a7b64b-0528-4c5c-a016-b3a4441e0484 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:00.655368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a7b64b-0528-4c5c-a016-b3a4441e0484 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:00.659726 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:00.661454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-01a7b64b-0528-4c5c-a016-b3a4441e0484 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:00.661454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:00.661454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:00.661454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:00.661454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:00.661454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:00.661454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:00.661454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:00.661454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:00.661454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:00.661454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:00.661454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:00.661454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:00.661454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:00.661454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:00.661454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:00.662824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:00.662824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:00.662824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:00.662824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:00.662824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:00.662824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:00.662824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:00.662824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:00.662824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:00.662824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:00.662824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:00.662824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:00.662824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:00.662824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:00.662824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:00.662824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:00.663594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:00.663594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:00.663594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:00.663594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:00.663594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:00.663594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:00.663594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:00.663594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:00.663594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:00.663594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:00.663594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:00.663594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:00.663594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 31/63] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:55:00.669207 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6603035-6d38-4d4c-b462-d899b21192e8 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:00.669533 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6603035-6d38-4d4c-b462-d899b21192e8 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:00.669888 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6603035-6d38-4d4c-b462-d899b21192e8 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:00.671355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 32/64] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:00.673062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6603035-6d38-4d4c-b462-d899b21192e8 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:00.725071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:00.729465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45667f3-44c3-42eb-ba14-16e2867eb994 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:00.766049 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386ea1e-7901-4be7-b628-8e30547d1f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:00.766938 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386ea1e-7901-4be7-b628-8e30547d1f15 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:00.767372 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386ea1e-7901-4be7-b628-8e30547d1f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:00.767789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386ea1e-7901-4be7-b628-8e30547d1f15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:00.781004 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3386ea1e-7901-4be7-b628-8e30547d1f15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:00.782596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6603035-6d38-4d4c-b462-d899b21192e8 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:00.788045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-3386ea1e-7901-4be7-b628-8e30547d1f15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:55:00.788045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:00.788045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:00.788045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:00.788045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:00.788045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:00.788045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:00.788045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:00.788045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:00.788045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:00.788045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:00.788045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:00.788045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:00.788045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:00.788045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:00.788045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:00.788945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:00.788945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:00.788945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:00.788945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:00.788945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:00.788945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:00.788945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:00.788945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:00.788945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:00.788945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:00.788945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:00.788945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:00.788945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:00.788945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:00.788945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:00.788945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:00.789719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:00.789719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:00.789719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:00.789719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:00.789719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:00.789719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:00.789719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:00.789719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:00.789719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:00.789719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:00.789719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:55:00.789719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:00.789719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 33/65] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Oct 22 23:55:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:55:00.794940 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cf503-4557-4999-b4aa-56ed69840e6d None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:00.795240 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cf503-4557-4999-b4aa-56ed69840e6d None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:00.795687 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62cf503-4557-4999-b4aa-56ed69840e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:00.814934 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-d62cf503-4557-4999-b4aa-56ed69840e6d None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:00.816135 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d62cf503-4557-4999-b4aa-56ed69840e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:00.824195 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab2b84-bf36-44ad-a670-cc1a0bbf7b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:00.825108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab2b84-bf36-44ad-a670-cc1a0bbf7b85 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:00.825397 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab2b84-bf36-44ad-a670-cc1a0bbf7b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:00.825677 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab2b84-bf36-44ad-a670-cc1a0bbf7b85 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:00.826632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab2b84-bf36-44ad-a670-cc1a0bbf7b85 None admin] RBAC: Adding query filter params (name=member) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:00.826945 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab2b84-bf36-44ad-a670-cc1a0bbf7b85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:00.828360 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab2b84-bf36-44ad-a670-cc1a0bbf7b85 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:00.837001 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 33/66] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Oct 22 23:55:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:00.837667 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab2b84-bf36-44ad-a670-cc1a0bbf7b85 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:00.884710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 34/67] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:00.885567 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cf503-4557-4999-b4aa-56ed69840e6d None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:00.919436 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:00.947316 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:00.960876 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab2b84-bf36-44ad-a670-cc1a0bbf7b85 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:00.995333 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62cf503-4557-4999-b4aa-56ed69840e6d None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:00.997675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5085fce-fd3b-4b3f-bda1-f0bbdbc2eb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:00.998320 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5085fce-fd3b-4b3f-bda1-f0bbdbc2eb49 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:00.998610 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5085fce-fd3b-4b3f-bda1-f0bbdbc2eb49 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:00.998893 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5085fce-fd3b-4b3f-bda1-f0bbdbc2eb49 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:01.009913 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5085fce-fd3b-4b3f-bda1-f0bbdbc2eb49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:01.015252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-e5085fce-fd3b-4b3f-bda1-f0bbdbc2eb49 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 22 23:55:01.015252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:01.015252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:01.015252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:01.015252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:01.015252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:01.015252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:01.015252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:01.015252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:01.015252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:01.015252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:01.015252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:01.015252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:01.015252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:01.015252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:01.015252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:01.016402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:01.016402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:01.016402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:01.016402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:01.016402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:01.016402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:01.016402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:01.016402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:01.016402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:01.016402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:01.016402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:01.016402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:01.016402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:01.016402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:01.016402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:01.016402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:01.017671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:01.017671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:01.017671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:01.017671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:01.017671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:01.017671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:01.017671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:01.017671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:01.017671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:01.017671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:01.017671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 22 23:55:01.017671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:01.017671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 34/68] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Oct 22 23:55:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:01.022081 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5085fce-fd3b-4b3f-bda1-f0bbdbc2eb49 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:01.033956 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd1c3e-f5a5-4627-acec-113dffa42f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:01.034571 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd1c3e-f5a5-4627-acec-113dffa42f42 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:01.034859 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd1c3e-f5a5-4627-acec-113dffa42f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:01.035135 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd1c3e-f5a5-4627-acec-113dffa42f42 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:01.035804 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd1c3e-f5a5-4627-acec-113dffa42f42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:01.036116 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd1c3e-f5a5-4627-acec-113dffa42f42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:01.037539 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd1c3e-f5a5-4627-acec-113dffa42f42 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:01.045845 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 35/69] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:01.046481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd1c3e-f5a5-4627-acec-113dffa42f42 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:01.132652 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5085fce-fd3b-4b3f-bda1-f0bbdbc2eb49 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:01.155290 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:01.162020 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd1c3e-f5a5-4627-acec-113dffa42f42 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:01.167319 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ded31-ce15-46b9-9add-2781f691f11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:01.167955 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ded31-ce15-46b9-9add-2781f691f11f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:01.168244 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ded31-ce15-46b9-9add-2781f691f11f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:01.168552 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ded31-ce15-46b9-9add-2781f691f11f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:01.179335 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123ded31-ce15-46b9-9add-2781f691f11f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:01.184785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-123ded31-ce15-46b9-9add-2781f691f11f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:01.184785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:01.184785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:01.184785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:01.184785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:01.184785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:01.184785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:01.184785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:01.184785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:01.184785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:01.184785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:01.184785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:01.184785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:01.184785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:01.184785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:01.184785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:01.185638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:01.185638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:01.185638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:01.185638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:01.185638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:01.185638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:01.185638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:01.185638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:01.185638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:01.185638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:01.185638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:01.185638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:01.185638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:01.185638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:01.185638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:01.185638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:01.186399 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:01.186399 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:01.186399 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:01.186399 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:01.186399 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:01.186399 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:01.186399 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:01.186399 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:01.186399 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:01.186399 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:01.186399 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:01.186399 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:01.186399 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 35/70] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:01.187167 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ded31-ce15-46b9-9add-2781f691f11f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:01.198765 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:01.199005 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3041b89-5fb3-4230-aafa-2ca82fdd55b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:01.199645 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3041b89-5fb3-4230-aafa-2ca82fdd55b5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:01.199932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3041b89-5fb3-4230-aafa-2ca82fdd55b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:01.200214 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3041b89-5fb3-4230-aafa-2ca82fdd55b5 None admin] PATH_INFO: `/v3/users/214a39c2cb9344d8bab53fdc6b5324b0` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:01.215678 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3041b89-5fb3-4230-aafa-2ca82fdd55b5 None admin] RBAC: Authorizing `identity:get_user(user_id=214a39c2cb9344d8bab53fdc6b5324b0)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:01.217171 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3041b89-5fb3-4230-aafa-2ca82fdd55b5 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:01.218832 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 36/71] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:55:01 2022] GET /identity/v3/users/214a39c2cb9344d8bab53fdc6b5324b0 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:01.219668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3041b89-5fb3-4230-aafa-2ca82fdd55b5 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:01.285217 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ded31-ce15-46b9-9add-2781f691f11f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:01.319432 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828b330-79ee-4c01-b10d-b2b96a0b6a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:01.320017 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3041b89-5fb3-4230-aafa-2ca82fdd55b5 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:01.320116 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828b330-79ee-4c01-b10d-b2b96a0b6a23 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:01.320328 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828b330-79ee-4c01-b10d-b2b96a0b6a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:01.320654 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6828b330-79ee-4c01-b10d-b2b96a0b6a23 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:01.321335 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828b330-79ee-4c01-b10d-b2b96a0b6a23 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:01.321650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828b330-79ee-4c01-b10d-b2b96a0b6a23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:01.323131 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6828b330-79ee-4c01-b10d-b2b96a0b6a23 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:01.331027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 36/72] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:55:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:01.331920 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828b330-79ee-4c01-b10d-b2b96a0b6a23 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:01.356553 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5760db-4bf0-4c49-aaac-12e097fac1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:01.357188 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5760db-4bf0-4c49-aaac-12e097fac1b3 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:01.357500 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5760db-4bf0-4c49-aaac-12e097fac1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:01.357778 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5760db-4bf0-4c49-aaac-12e097fac1b3 None admin] PATH_INFO: `/v3/users/ac8d80b2c4a24dfea25c6de809bc37e3` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:01.358917 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5760db-4bf0-4c49-aaac-12e097fac1b3 None admin] RBAC: Authorizing `identity:get_user(user_id=ac8d80b2c4a24dfea25c6de809bc37e3)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:01.361400 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5760db-4bf0-4c49-aaac-12e097fac1b3 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:01.362976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 37/73] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:55:01 2022] GET /identity/v3/users/ac8d80b2c4a24dfea25c6de809bc37e3 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:01.363719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5760db-4bf0-4c49-aaac-12e097fac1b3 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:01.389297 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:01.401888 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:01.432468 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6828b330-79ee-4c01-b10d-b2b96a0b6a23 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:01.466986 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f4199-6d85-4357-8ebf-8c8992affb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:01.467618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f4199-6d85-4357-8ebf-8c8992affb9f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:01.467901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f4199-6d85-4357-8ebf-8c8992affb9f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:01.468183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f4199-6d85-4357-8ebf-8c8992affb9f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:01.469767 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f4199-6d85-4357-8ebf-8c8992affb9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:01.470208 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5760db-4bf0-4c49-aaac-12e097fac1b3 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:01.471199 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f4199-6d85-4357-8ebf-8c8992affb9f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:01.473000 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 37/74] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:01.473679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f4199-6d85-4357-8ebf-8c8992affb9f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:01.504151 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919665b-f4c3-40f9-a449-16418d8597a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:01.504800 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919665b-f4c3-40f9-a449-16418d8597a1 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:01.505095 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919665b-f4c3-40f9-a449-16418d8597a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:01.505383 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919665b-f4c3-40f9-a449-16418d8597a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:01.506054 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5919665b-f4c3-40f9-a449-16418d8597a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:01.506363 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5919665b-f4c3-40f9-a449-16418d8597a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:01.507755 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5919665b-f4c3-40f9-a449-16418d8597a1 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:01.516477 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 38/75] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Oct 22 23:55:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:01.517297 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919665b-f4c3-40f9-a449-16418d8597a1 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:01.575767 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f4199-6d85-4357-8ebf-8c8992affb9f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:01.601789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:01.609636 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac899a-c844-4a4e-9767-0ef93ad3bec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:01.610271 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac899a-c844-4a4e-9767-0ef93ad3bec7 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:01.610559 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac899a-c844-4a4e-9767-0ef93ad3bec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:01.610841 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac899a-c844-4a4e-9767-0ef93ad3bec7 None admin] PATH_INFO: `/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:01.611961 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac899a-c844-4a4e-9767-0ef93ad3bec7 None admin] RBAC: Authorizing `identity:get_project(project_id=4d6b66c470f74a4dac6f66e6fcee2a51)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:01.613447 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac899a-c844-4a4e-9767-0ef93ad3bec7 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:01.615081 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 38/76] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:55:01 2022] GET /identity/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:01.615856 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac899a-c844-4a4e-9767-0ef93ad3bec7 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:01.617553 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919665b-f4c3-40f9-a449-16418d8597a1 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:01.626900 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:01.654237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a2f27-0145-4713-bf69-3e94701454b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:01.654867 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a2f27-0145-4713-bf69-3e94701454b7 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:01.655153 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a2f27-0145-4713-bf69-3e94701454b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:01.655431 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a2f27-0145-4713-bf69-3e94701454b7 None admin] PATH_INFO: `/v3/users/214a39c2cb9344d8bab53fdc6b5324b0` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:01.656590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a2f27-0145-4713-bf69-3e94701454b7 None admin] RBAC: Authorizing `identity:get_user(user_id=214a39c2cb9344d8bab53fdc6b5324b0)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:01.658031 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a2f27-0145-4713-bf69-3e94701454b7 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:01.659597 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 39/77] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:55:01 2022] GET /identity/v3/users/214a39c2cb9344d8bab53fdc6b5324b0 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:01.660347 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a2f27-0145-4713-bf69-3e94701454b7 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:01.715854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac899a-c844-4a4e-9767-0ef93ad3bec7 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:01.748894 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48b8ad-0d92-4349-988a-be4bac33491d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:01.749549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48b8ad-0d92-4349-988a-be4bac33491d None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:01.749839 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48b8ad-0d92-4349-988a-be4bac33491d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:01.750658 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e48b8ad-0d92-4349-988a-be4bac33491d None admin] PATH_INFO: `/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:01.751788 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e48b8ad-0d92-4349-988a-be4bac33491d None admin] RBAC: Authorizing `identity:get_project(project_id=4d6b66c470f74a4dac6f66e6fcee2a51)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:01.753279 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e48b8ad-0d92-4349-988a-be4bac33491d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:01.754909 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 39/78] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:55:01 2022] GET /identity/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:01.755686 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48b8ad-0d92-4349-988a-be4bac33491d None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:01.765524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a2f27-0145-4713-bf69-3e94701454b7 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:01.800365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e762b9-cba8-40d3-9a67-5a31c4ec3f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:01.801020 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e762b9-cba8-40d3-9a67-5a31c4ec3f8a None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:01.801306 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e762b9-cba8-40d3-9a67-5a31c4ec3f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:01.801581 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e762b9-cba8-40d3-9a67-5a31c4ec3f8a None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:01.802229 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e762b9-cba8-40d3-9a67-5a31c4ec3f8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:01.803648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e762b9-cba8-40d3-9a67-5a31c4ec3f8a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:01.830019 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:01.832320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 40/79] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Oct 22 23:55:01 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:55:01.844745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e762b9-cba8-40d3-9a67-5a31c4ec3f8a None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:01.849226 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:01.864412 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e48b8ad-0d92-4349-988a-be4bac33491d None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:01.901300 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de7446-3a1a-4e22-b31a-d890e2c14113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:01.901944 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de7446-3a1a-4e22-b31a-d890e2c14113 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:01.902232 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de7446-3a1a-4e22-b31a-d890e2c14113 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:01.902510 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de7446-3a1a-4e22-b31a-d890e2c14113 None admin] PATH_INFO: `/v3/users/214a39c2cb9344d8bab53fdc6b5324b0` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:01.903679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de7446-3a1a-4e22-b31a-d890e2c14113 None admin] RBAC: Authorizing `identity:get_user(user_id=214a39c2cb9344d8bab53fdc6b5324b0)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:01.905220 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de7446-3a1a-4e22-b31a-d890e2c14113 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:01.907648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 40/80] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:55:01 2022] GET /identity/v3/users/214a39c2cb9344d8bab53fdc6b5324b0 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:01.908382 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de7446-3a1a-4e22-b31a-d890e2c14113 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:01.965445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e762b9-cba8-40d3-9a67-5a31c4ec3f8a None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:02.006712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9593f8-7c9e-440f-81c5-442f3aa96a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:02.007713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9593f8-7c9e-440f-81c5-442f3aa96a46 None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:02.008010 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9593f8-7c9e-440f-81c5-442f3aa96a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:02.008314 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9593f8-7c9e-440f-81c5-442f3aa96a46 None admin] PATH_INFO: `/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51/users/214a39c2cb9344d8bab53fdc6b5324b0/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:02.010501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9593f8-7c9e-440f-81c5-442f3aa96a46 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d6b66c470f74a4dac6f66e6fcee2a51, user_id=214a39c2cb9344d8bab53fdc6b5324b0, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:02.012026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9593f8-7c9e-440f-81c5-442f3aa96a46 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:02.017447 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de7446-3a1a-4e22-b31a-d890e2c14113 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:02.030918 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 41/81] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:55:01 2022] PUT /identity/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51/users/214a39c2cb9344d8bab53fdc6b5324b0/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:55:02.032755 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9593f8-7c9e-440f-81c5-442f3aa96a46 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:02.061391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9be7f-1b00-491a-9934-2cc62d471630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:02.062040 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9be7f-1b00-491a-9934-2cc62d471630 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:02.063758 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9be7f-1b00-491a-9934-2cc62d471630 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:02.064107 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9be7f-1b00-491a-9934-2cc62d471630 None admin] PATH_INFO: `/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:02.069845 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b9be7f-1b00-491a-9934-2cc62d471630 None admin] RBAC: Authorizing `identity:get_project(project_id=4d6b66c470f74a4dac6f66e6fcee2a51)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:02.071475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b9be7f-1b00-491a-9934-2cc62d471630 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:02.073551 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 41/82] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:55:01 2022] GET /identity/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:02.074492 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9be7f-1b00-491a-9934-2cc62d471630 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:02.090839 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:02.150775 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:02.207954 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9593f8-7c9e-440f-81c5-442f3aa96a46 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:02.236073 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9be7f-1b00-491a-9934-2cc62d471630 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:02.260662 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03be0aa-ff96-4acb-b68c-c008a5ff2c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:02.261337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03be0aa-ff96-4acb-b68c-c008a5ff2c31 None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:02.261624 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03be0aa-ff96-4acb-b68c-c008a5ff2c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:02.261901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03be0aa-ff96-4acb-b68c-c008a5ff2c31 None admin] PATH_INFO: `/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51/users/ac8d80b2c4a24dfea25c6de809bc37e3/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:02.264839 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03be0aa-ff96-4acb-b68c-c008a5ff2c31 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d6b66c470f74a4dac6f66e6fcee2a51, user_id=ac8d80b2c4a24dfea25c6de809bc37e3, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:02.267392 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03be0aa-ff96-4acb-b68c-c008a5ff2c31 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:02.282610 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330ac5c-e5c6-4fa7-81ba-ef59903340cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:02.283263 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330ac5c-e5c6-4fa7-81ba-ef59903340cd None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:02.283548 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330ac5c-e5c6-4fa7-81ba-ef59903340cd None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:02.283886 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330ac5c-e5c6-4fa7-81ba-ef59903340cd None admin] PATH_INFO: `/v3/projects/aa6452edd45c426e9eb7e0a891ba2952` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:02.284577 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 42/83] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:55:02 2022] PUT /identity/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51/users/ac8d80b2c4a24dfea25c6de809bc37e3/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:55:02.285649 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03be0aa-ff96-4acb-b68c-c008a5ff2c31 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:02.285881 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0330ac5c-e5c6-4fa7-81ba-ef59903340cd None admin] RBAC: Authorizing `identity:get_project(project_id=aa6452edd45c426e9eb7e0a891ba2952)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:02.287587 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0330ac5c-e5c6-4fa7-81ba-ef59903340cd None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:02.289443 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 42/84] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:55:02 2022] GET /identity/v3/projects/aa6452edd45c426e9eb7e0a891ba2952 => generated 312 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:02.295711 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330ac5c-e5c6-4fa7-81ba-ef59903340cd None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:02.409966 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03be0aa-ff96-4acb-b68c-c008a5ff2c31 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:02.432905 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:02.463101 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:02.478810 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330ac5c-e5c6-4fa7-81ba-ef59903340cd None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:02.492799 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05fd008-bc4c-4497-937c-3e51628b0c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:02.493639 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05fd008-bc4c-4497-937c-3e51628b0c87 None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:02.493933 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05fd008-bc4c-4497-937c-3e51628b0c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:02.494219 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05fd008-bc4c-4497-937c-3e51628b0c87 None admin] PATH_INFO: `/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51/users/214a39c2cb9344d8bab53fdc6b5324b0/roles/cf8116e4e82748ac872d8763d113ba61` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:02.506426 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05fd008-bc4c-4497-937c-3e51628b0c87 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d6b66c470f74a4dac6f66e6fcee2a51, user_id=214a39c2cb9344d8bab53fdc6b5324b0, role_id=cf8116e4e82748ac872d8763d113ba61)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:02.506700 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05fd008-bc4c-4497-937c-3e51628b0c87 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:02.531282 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 43/85] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:55:02 2022] PUT /identity/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51/users/214a39c2cb9344d8bab53fdc6b5324b0/roles/cf8116e4e82748ac872d8763d113ba61 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:55:02.563610 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54e822-a36e-4b31-b8a2-6c0ab9cb5485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:02.564304 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54e822-a36e-4b31-b8a2-6c0ab9cb5485 None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:02.564616 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54e822-a36e-4b31-b8a2-6c0ab9cb5485 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:02.565152 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54e822-a36e-4b31-b8a2-6c0ab9cb5485 None admin] PATH_INFO: `/v3/projects/aa6452edd45c426e9eb7e0a891ba2952/users/214a39c2cb9344d8bab53fdc6b5324b0/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:02.567198 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54e822-a36e-4b31-b8a2-6c0ab9cb5485 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa6452edd45c426e9eb7e0a891ba2952, user_id=214a39c2cb9344d8bab53fdc6b5324b0, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:02.568722 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54e822-a36e-4b31-b8a2-6c0ab9cb5485 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:02.590490 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 43/86] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:55:02 2022] PUT /identity/v3/projects/aa6452edd45c426e9eb7e0a891ba2952/users/214a39c2cb9344d8bab53fdc6b5324b0/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 22 23:55:04.944097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831824ea-d37f-4629-a89d-fa9295607698 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:04.944097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831824ea-d37f-4629-a89d-fa9295607698 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:04.944097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831824ea-d37f-4629-a89d-fa9295607698 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:04.944097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 44/87] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 22 23:55:04.949495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fd68d-69c7-4fa7-9954-e5f7a4d2e897 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:04.950294 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fd68d-69c7-4fa7-9954-e5f7a4d2e897 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:04.950776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fd68d-69c7-4fa7-9954-e5f7a4d2e897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:04.977853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-d94fd68d-69c7-4fa7-9954-e5f7a4d2e897 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:04.979392 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d94fd68d-69c7-4fa7-9954-e5f7a4d2e897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:05.073148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 44/88] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:05.184168 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96558c8a-aba5-404f-ba21-e7b12f3c04e5 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:05.184841 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96558c8a-aba5-404f-ba21-e7b12f3c04e5 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:05.185328 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96558c8a-aba5-404f-ba21-e7b12f3c04e5 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:05.186954 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 45/89] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:05.193062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a4720-403a-42a9-9f0e-fe86cdeeb73b None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:05.193607 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a4720-403a-42a9-9f0e-fe86cdeeb73b None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:05.194172 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a4720-403a-42a9-9f0e-fe86cdeeb73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:05.221842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-106a4720-403a-42a9-9f0e-fe86cdeeb73b None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:05.223434 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-106a4720-403a-42a9-9f0e-fe86cdeeb73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:05.314628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:05.322407 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 45/90] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:05.416163 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb488f05-130e-4f6e-9832-4775da86d9bf None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:05.416846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb488f05-130e-4f6e-9832-4775da86d9bf None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:05.417330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb488f05-130e-4f6e-9832-4775da86d9bf None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:05.418872 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 46/91] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:05.424517 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edee920-b02a-49d6-b163-fef6f77944a7 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:05.425045 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edee920-b02a-49d6-b163-fef6f77944a7 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:05.425554 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edee920-b02a-49d6-b163-fef6f77944a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:05.461613 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-1edee920-b02a-49d6-b163-fef6f77944a7 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:05.463124 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-1edee920-b02a-49d6-b163-fef6f77944a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:05.533627 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb488f05-130e-4f6e-9832-4775da86d9bf None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:05.611330 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 46/92] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 22 23:55:05.682138 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb488f05-130e-4f6e-9832-4775da86d9bf None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:05.688899 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f072fd-c79a-46b9-bc20-04906a1f2639 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:05.689193 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f072fd-c79a-46b9-bc20-04906a1f2639 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:05.689468 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f072fd-c79a-46b9-bc20-04906a1f2639 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:05.691097 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 47/93] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 22 23:55:05.696733 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88bb49-60fc-46be-a930-f5fac09643f8 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:05.697019 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88bb49-60fc-46be-a930-f5fac09643f8 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:05.697301 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88bb49-60fc-46be-a930-f5fac09643f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:05.736863 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-9b88bb49-60fc-46be-a930-f5fac09643f8 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:05.737998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9b88bb49-60fc-46be-a930-f5fac09643f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:05.763678 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc5c9b-dda6-4124-9bea-1e2070bb404d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:05.764322 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc5c9b-dda6-4124-9bea-1e2070bb404d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:05.772700 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc5c9b-dda6-4124-9bea-1e2070bb404d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:05.773011 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc5c9b-dda6-4124-9bea-1e2070bb404d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:05.793484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc5c9b-dda6-4124-9bea-1e2070bb404d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:05.799115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-8efc5c9b-dda6-4124-9bea-1e2070bb404d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:05.799115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:05.799115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:05.799115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:05.799115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:05.799115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:05.799115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:05.799115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:05.799115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:05.799115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:05.799115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:05.799115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:05.799115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:05.799115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:05.799115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:05.799115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:05.799972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:05.799972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:05.799972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:05.799972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:05.799972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:05.799972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:05.799972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:05.799972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:05.799972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:05.799972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:05.799972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:05.799972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:05.799972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:05.799972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:05.799972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:05.799972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:05.800854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:05.800854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:05.800854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:05.800854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:05.800854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:05.800854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:05.800854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:05.800854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:05.800854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:05.800854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:05.800854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:05.800854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:05.800854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 47/94] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:55:05.801499 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc5c9b-dda6-4124-9bea-1e2070bb404d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:05.860047 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 48/95] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:05.861485 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b88bb49-60fc-46be-a930-f5fac09643f8 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:05.864620 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:05.933478 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:05.958244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc5c9b-dda6-4124-9bea-1e2070bb404d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:05.995509 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc692a-e645-44c7-8e12-6c8911941d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:05.996466 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc692a-e645-44c7-8e12-6c8911941d1e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:05.997020 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc692a-e645-44c7-8e12-6c8911941d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:05.997584 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc692a-e645-44c7-8e12-6c8911941d1e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:06.009754 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fc692a-e645-44c7-8e12-6c8911941d1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:06.015157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-66fc692a-e645-44c7-8e12-6c8911941d1e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 22 23:55:06.015157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:06.015157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:06.015157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:06.015157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:06.015157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:06.015157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:06.015157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:06.015157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:06.015157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:06.015157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:06.015157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:06.015157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:06.015157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:06.015157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:06.015157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:06.016710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:06.016710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:06.016710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:06.016710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:06.016710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:06.016710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:06.016710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:06.016710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:06.016710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:06.016710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:06.016710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:06.016710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:06.016710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:06.016710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:06.016710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:06.016710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:06.017590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:06.017590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:06.017590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:06.017590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:06.017590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:06.017590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:06.017590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:06.017590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:06.017590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:06.017590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:06.017590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 22 23:55:06.017590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:06.024596 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 48/96] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:06.026347 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cf1cb-b58d-4ece-a61d-1241a41a3621 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:06.026833 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cf1cb-b58d-4ece-a61d-1241a41a3621 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:06.027304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0cf1cb-b58d-4ece-a61d-1241a41a3621 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:06.028203 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b88bb49-60fc-46be-a930-f5fac09643f8 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:06.030067 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 49/97] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:06.031763 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0cf1cb-b58d-4ece-a61d-1241a41a3621 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:06.114342 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9cdd4d-1c73-46c6-9e23-4a295afe48f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:06.114984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9cdd4d-1c73-46c6-9e23-4a295afe48f4 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:06.115631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9cdd4d-1c73-46c6-9e23-4a295afe48f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:06.115631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9cdd4d-1c73-46c6-9e23-4a295afe48f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:06.116851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9cdd4d-1c73-46c6-9e23-4a295afe48f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:06.117650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9cdd4d-1c73-46c6-9e23-4a295afe48f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:06.119034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9cdd4d-1c73-46c6-9e23-4a295afe48f4 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:06.129366 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 49/98] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Oct 22 23:55:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:06.130210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57444ba0-728d-44d0-87ff-caad39964c50 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:06.132694 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57444ba0-728d-44d0-87ff-caad39964c50 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:06.133025 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57444ba0-728d-44d0-87ff-caad39964c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:06.149587 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0cf1cb-b58d-4ece-a61d-1241a41a3621 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:06.155145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-57444ba0-728d-44d0-87ff-caad39964c50 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:06.156420 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-57444ba0-728d-44d0-87ff-caad39964c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:06.160795 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:06.190264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671cd7f-1dcd-47b2-b073-2db353a55079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:06.190885 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671cd7f-1dcd-47b2-b073-2db353a55079 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:06.191172 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671cd7f-1dcd-47b2-b073-2db353a55079 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:06.191450 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671cd7f-1dcd-47b2-b073-2db353a55079 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:06.192118 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671cd7f-1dcd-47b2-b073-2db353a55079 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:06.192425 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671cd7f-1dcd-47b2-b073-2db353a55079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:06.193846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671cd7f-1dcd-47b2-b073-2db353a55079 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:06.202368 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 50/99] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Oct 22 23:55:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:55:06.202971 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671cd7f-1dcd-47b2-b073-2db353a55079 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:06.228145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 50/100] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:06.229474 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57444ba0-728d-44d0-87ff-caad39964c50 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:06.298143 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:06.332714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671cd7f-1dcd-47b2-b073-2db353a55079 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:06.363761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57444ba0-728d-44d0-87ff-caad39964c50 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:06.375021 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4fc10-8dde-45f1-be70-ee0c11d7f84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:06.376193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4fc10-8dde-45f1-be70-ee0c11d7f84c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:06.376505 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4fc10-8dde-45f1-be70-ee0c11d7f84c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:06.376797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4fc10-8dde-45f1-be70-ee0c11d7f84c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:06.391503 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da4fc10-8dde-45f1-be70-ee0c11d7f84c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:06.397639 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-9da4fc10-8dde-45f1-be70-ee0c11d7f84c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:55:06.397639 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:06.397639 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:06.397639 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:06.397639 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:06.397639 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:06.397639 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:06.397639 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:06.397639 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:06.397639 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:06.397639 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:06.397639 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:06.397639 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:06.397639 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:06.397639 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:06.397639 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:06.398484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:06.398484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:06.398484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:06.398484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:06.398484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:06.398484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:06.398484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:06.398484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:06.398484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:06.398484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:06.398484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:06.398484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:06.398484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:06.398484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:06.398484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:06.398484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:06.399259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:06.399259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:06.399259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:06.399259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:06.399259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:06.399259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:06.399259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:06.399259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:06.399259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:06.399259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:06.399259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:55:06.399259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:06.401322 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 51/101] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Oct 22 23:55:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:06.402232 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4fc10-8dde-45f1-be70-ee0c11d7f84c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:06.414437 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeaed7a-4927-4edf-aa11-d3ed48cd2890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:06.415095 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeaed7a-4927-4edf-aa11-d3ed48cd2890 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:06.415384 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeaed7a-4927-4edf-aa11-d3ed48cd2890 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:06.415673 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeaed7a-4927-4edf-aa11-d3ed48cd2890 None admin] PATH_INFO: `/v3/users/214a39c2cb9344d8bab53fdc6b5324b0` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:06.417252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeaed7a-4927-4edf-aa11-d3ed48cd2890 None admin] RBAC: Authorizing `identity:get_user(user_id=214a39c2cb9344d8bab53fdc6b5324b0)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:06.419408 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeaed7a-4927-4edf-aa11-d3ed48cd2890 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:06.424622 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 51/102] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:55:06 2022] GET /identity/v3/users/214a39c2cb9344d8bab53fdc6b5324b0 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:06.425604 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeaed7a-4927-4edf-aa11-d3ed48cd2890 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:06.442340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:06.521213 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4fc10-8dde-45f1-be70-ee0c11d7f84c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:06.536670 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeaed7a-4927-4edf-aa11-d3ed48cd2890 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:06.553461 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:06.557221 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109de4fa-5160-4462-a9d6-6754c31cc943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:06.557846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109de4fa-5160-4462-a9d6-6754c31cc943 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:06.558134 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109de4fa-5160-4462-a9d6-6754c31cc943 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:06.558405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109de4fa-5160-4462-a9d6-6754c31cc943 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:06.568742 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109de4fa-5160-4462-a9d6-6754c31cc943 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:06.570749 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781258e-4f20-4a58-ac3b-4e5055d49684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:06.571377 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781258e-4f20-4a58-ac3b-4e5055d49684 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:06.571665 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781258e-4f20-4a58-ac3b-4e5055d49684 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:06.571946 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781258e-4f20-4a58-ac3b-4e5055d49684 None admin] PATH_INFO: `/v3/users/4a0fb7285e844f7a8d0e53f7b6814786` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:06.573536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-109de4fa-5160-4462-a9d6-6754c31cc943 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 22 23:55:06.573536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:06.573536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:06.573536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:06.573536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:06.573536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:06.573536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:06.573536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:06.573536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:06.573536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:06.573536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:06.573536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:06.573536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:06.573536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:06.573536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:06.573536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:06.574516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:06.574516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:06.574516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:06.574516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:06.574516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:06.574516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:06.574516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:06.574516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:06.574516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:06.574516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:06.574516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:06.574516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:06.574516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:06.574516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:06.574516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:06.574516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:06.575481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:06.575481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:06.575481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:06.575481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:06.575481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:06.575481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:06.575481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:06.575481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:06.575481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:06.575481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:06.575481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 22 23:55:06.575481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:06.575481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 52/103] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Oct 22 23:55:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:06.575481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109de4fa-5160-4462-a9d6-6754c31cc943 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:06.590225 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781258e-4f20-4a58-ac3b-4e5055d49684 None admin] RBAC: Authorizing `identity:get_user(user_id=4a0fb7285e844f7a8d0e53f7b6814786)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:06.591704 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781258e-4f20-4a58-ac3b-4e5055d49684 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:06.593694 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 52/104] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:55:06 2022] GET /identity/v3/users/4a0fb7285e844f7a8d0e53f7b6814786 => generated 289 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:06.595159 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781258e-4f20-4a58-ac3b-4e5055d49684 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:06.668042 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:06.678154 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109de4fa-5160-4462-a9d6-6754c31cc943 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:06.693466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781258e-4f20-4a58-ac3b-4e5055d49684 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:06.712225 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8c6b9c-cd2f-4fb2-8f8d-0affd3a8433d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:06.712858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c6b9c-cd2f-4fb2-8f8d-0affd3a8433d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:06.713156 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c6b9c-cd2f-4fb2-8f8d-0affd3a8433d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:06.713437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8c6b9c-cd2f-4fb2-8f8d-0affd3a8433d None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:06.714095 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8c6b9c-cd2f-4fb2-8f8d-0affd3a8433d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:06.714402 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8c6b9c-cd2f-4fb2-8f8d-0affd3a8433d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:06.715783 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8c6b9c-cd2f-4fb2-8f8d-0affd3a8433d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:06.723702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 53/105] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:06.725763 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e67eaf-4cac-4aae-8872-25b1413d57f1 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:06.726062 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e67eaf-4cac-4aae-8872-25b1413d57f1 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:06.726347 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e67eaf-4cac-4aae-8872-25b1413d57f1 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:06.726853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c988c-9e6b-4d9d-a686-37d6855f70d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:06.727453 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c988c-9e6b-4d9d-a686-37d6855f70d2 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:06.728557 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 54/106] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:06.728666 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c988c-9e6b-4d9d-a686-37d6855f70d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:06.728666 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c988c-9e6b-4d9d-a686-37d6855f70d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:06.728813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e67eaf-4cac-4aae-8872-25b1413d57f1 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:06.776838 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:06.801051 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c988c-9e6b-4d9d-a686-37d6855f70d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:06.802535 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c988c-9e6b-4d9d-a686-37d6855f70d2 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:06.833771 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e67eaf-4cac-4aae-8872-25b1413d57f1 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:06.866977 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6e555-9084-42cb-80f3-e3ba6d3b8e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:06.867601 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6e555-9084-42cb-80f3-e3ba6d3b8e33 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:06.867888 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6e555-9084-42cb-80f3-e3ba6d3b8e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:06.868172 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6e555-9084-42cb-80f3-e3ba6d3b8e33 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:06.869417 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6e555-9084-42cb-80f3-e3ba6d3b8e33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:06.870849 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6e555-9084-42cb-80f3-e3ba6d3b8e33 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:06.872599 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 55/107] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:06.873201 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6e555-9084-42cb-80f3-e3ba6d3b8e33 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:06.894794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:06.902020 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 53/108] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:55:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 22 23:55:06.902020 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c988c-9e6b-4d9d-a686-37d6855f70d2 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:06.975288 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:06.978196 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6e555-9084-42cb-80f3-e3ba6d3b8e33 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:06.998812 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c988c-9e6b-4d9d-a686-37d6855f70d2 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:07.011537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c6abc-ca62-4c59-881c-89a0117a86d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:07.012148 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c6abc-ca62-4c59-881c-89a0117a86d8 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:07.012433 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c6abc-ca62-4c59-881c-89a0117a86d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:07.012740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c6abc-ca62-4c59-881c-89a0117a86d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:07.013395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c6abc-ca62-4c59-881c-89a0117a86d8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:07.013704 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c6abc-ca62-4c59-881c-89a0117a86d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:07.015078 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c6abc-ca62-4c59-881c-89a0117a86d8 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:07.023177 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 56/109] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:55:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:07.023969 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c6abc-ca62-4c59-881c-89a0117a86d8 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:07.036358 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed82c51-ebb1-4e33-8e7b-7ed61cd99ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:07.037029 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed82c51-ebb1-4e33-8e7b-7ed61cd99ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:07.037316 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed82c51-ebb1-4e33-8e7b-7ed61cd99ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:07.037598 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed82c51-ebb1-4e33-8e7b-7ed61cd99ff5 None admin] PATH_INFO: `/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:07.038736 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed82c51-ebb1-4e33-8e7b-7ed61cd99ff5 None admin] RBAC: Authorizing `identity:get_project(project_id=4d6b66c470f74a4dac6f66e6fcee2a51)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:07.040175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed82c51-ebb1-4e33-8e7b-7ed61cd99ff5 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:07.041840 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 54/110] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:55:06 2022] GET /identity/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:07.043011 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5513bb9-6815-4fa9-a854-de6ed4694119 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:07.043304 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5513bb9-6815-4fa9-a854-de6ed4694119 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:07.043584 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5513bb9-6815-4fa9-a854-de6ed4694119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:07.062052 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-a5513bb9-6815-4fa9-a854-de6ed4694119 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:07.063456 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a5513bb9-6815-4fa9-a854-de6ed4694119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:07.109039 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:07.127817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c6abc-ca62-4c59-881c-89a0117a86d8 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:07.129640 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 55/111] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:07.134791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5513bb9-6815-4fa9-a854-de6ed4694119 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:07.166612 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe7205-0c82-495c-9e42-026dbb11a32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:07.167594 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe7205-0c82-495c-9e42-026dbb11a32f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:07.168009 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe7205-0c82-495c-9e42-026dbb11a32f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:07.168359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe7205-0c82-495c-9e42-026dbb11a32f None admin] PATH_INFO: `/v3/users/ac8d80b2c4a24dfea25c6de809bc37e3` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:07.169678 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbe7205-0c82-495c-9e42-026dbb11a32f None admin] RBAC: Authorizing `identity:get_user(user_id=ac8d80b2c4a24dfea25c6de809bc37e3)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:07.171339 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbe7205-0c82-495c-9e42-026dbb11a32f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:07.178397 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 57/112] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:55:07 2022] GET /identity/v3/users/ac8d80b2c4a24dfea25c6de809bc37e3 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:07.178397 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe7205-0c82-495c-9e42-026dbb11a32f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:07.214451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:07.240246 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5513bb9-6815-4fa9-a854-de6ed4694119 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:07.274599 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de43a802-1c5d-493d-9022-b088f033cd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:07.275218 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43a802-1c5d-493d-9022-b088f033cd78 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:07.275509 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43a802-1c5d-493d-9022-b088f033cd78 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:07.275793 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43a802-1c5d-493d-9022-b088f033cd78 None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:07.276451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de43a802-1c5d-493d-9022-b088f033cd78 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:07.277916 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de43a802-1c5d-493d-9022-b088f033cd78 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:07.293522 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe7205-0c82-495c-9e42-026dbb11a32f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:07.305643 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 56/113] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Oct 22 23:55:07 2022] POST /identity/v3/projects => generated 302 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:55:07.306754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de43a802-1c5d-493d-9022-b088f033cd78 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:07.359497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f9fc14-4eab-48b8-8840-59e247de271a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:07.360103 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9fc14-4eab-48b8-8840-59e247de271a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:07.360397 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9fc14-4eab-48b8-8840-59e247de271a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:07.360721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f9fc14-4eab-48b8-8840-59e247de271a None admin] PATH_INFO: `/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:07.361937 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f9fc14-4eab-48b8-8840-59e247de271a None admin] RBAC: Authorizing `identity:get_project(project_id=4d6b66c470f74a4dac6f66e6fcee2a51)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:07.364155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f9fc14-4eab-48b8-8840-59e247de271a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:07.367270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 58/114] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:55:07 2022] GET /identity/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:07.368206 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f9fc14-4eab-48b8-8840-59e247de271a None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:07.402019 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:07.438033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de43a802-1c5d-493d-9022-b088f033cd78 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:07.467948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:07.479111 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fff982-252e-4cd7-aadb-422f10ddb642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:07.479773 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fff982-252e-4cd7-aadb-422f10ddb642 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:07.480066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fff982-252e-4cd7-aadb-422f10ddb642 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:07.480412 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fff982-252e-4cd7-aadb-422f10ddb642 None admin] PATH_INFO: `/v3/users/214a39c2cb9344d8bab53fdc6b5324b0` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:07.481637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fff982-252e-4cd7-aadb-422f10ddb642 None admin] RBAC: Authorizing `identity:get_user(user_id=214a39c2cb9344d8bab53fdc6b5324b0)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:07.483123 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fff982-252e-4cd7-aadb-422f10ddb642 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:07.489443 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 57/115] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:55:07 2022] GET /identity/v3/users/214a39c2cb9344d8bab53fdc6b5324b0 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:07.489443 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fff982-252e-4cd7-aadb-422f10ddb642 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:07.504693 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f9fc14-4eab-48b8-8840-59e247de271a None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:07.547219 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d95f8-b221-431f-96d5-7c39e6e04e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:07.547898 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d95f8-b221-431f-96d5-7c39e6e04e2f None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:07.548190 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d95f8-b221-431f-96d5-7c39e6e04e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:07.548478 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d95f8-b221-431f-96d5-7c39e6e04e2f None admin] PATH_INFO: `/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51/users/4a0fb7285e844f7a8d0e53f7b6814786/roles/159c6380cd984e998bb6a97ed7d98dac` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:07.550590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d95f8-b221-431f-96d5-7c39e6e04e2f None admin] RBAC: Authorizing `identity:create_grant(project_id=4d6b66c470f74a4dac6f66e6fcee2a51, user_id=4a0fb7285e844f7a8d0e53f7b6814786, role_id=159c6380cd984e998bb6a97ed7d98dac)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:07.552047 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d95f8-b221-431f-96d5-7c39e6e04e2f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:07.567133 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 59/116] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:55:07 2022] PUT /identity/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51/users/4a0fb7285e844f7a8d0e53f7b6814786/roles/159c6380cd984e998bb6a97ed7d98dac => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:55:07.569348 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d95f8-b221-431f-96d5-7c39e6e04e2f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:07.613790 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fff982-252e-4cd7-aadb-422f10ddb642 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:07.666674 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ba3de-b52a-407b-8a1d-b373d2ffbcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:07.668660 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ba3de-b52a-407b-8a1d-b373d2ffbcfb None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:07.669137 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ba3de-b52a-407b-8a1d-b373d2ffbcfb None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:07.669570 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ba3de-b52a-407b-8a1d-b373d2ffbcfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:07.677997 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:07.716956 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d95f8-b221-431f-96d5-7c39e6e04e2f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:07.762009 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5fc32-eec0-448a-84ee-d24e1846a94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:07.763039 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5fc32-eec0-448a-84ee-d24e1846a94f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:07.763476 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5fc32-eec0-448a-84ee-d24e1846a94f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:07.764190 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5fc32-eec0-448a-84ee-d24e1846a94f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:07.765124 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f5fc32-eec0-448a-84ee-d24e1846a94f None admin] RBAC: Adding query filter params (role.id=4bd2874c95634a50aac9fccbddc7c674, scope.project.id=4d6b66c470f74a4dac6f66e6fcee2a51, user.id=214a39c2cb9344d8bab53fdc6b5324b0) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:07.765588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f5fc32-eec0-448a-84ee-d24e1846a94f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:07.767652 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f5fc32-eec0-448a-84ee-d24e1846a94f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:07.777394 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 60/117] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:55:07 2022] GET /identity/v3/role_assignments?user.id=214a39c2cb9344d8bab53fdc6b5324b0&scope.project.id=4d6b66c470f74a4dac6f66e6fcee2a51&role.id=4bd2874c95634a50aac9fccbddc7c674 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:07.778614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5fc32-eec0-448a-84ee-d24e1846a94f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:07.787426 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:07.798819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ba3de-b52a-407b-8a1d-b373d2ffbcfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:07.800315 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ba3de-b52a-407b-8a1d-b373d2ffbcfb None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:07.912556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5fc32-eec0-448a-84ee-d24e1846a94f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:07.977343 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a49ef-e64b-402d-ae32-66e3725e4335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:07.978013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a49ef-e64b-402d-ae32-66e3725e4335 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:07.978306 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a49ef-e64b-402d-ae32-66e3725e4335 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:07.978854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a49ef-e64b-402d-ae32-66e3725e4335 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:08.001980 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a49ef-e64b-402d-ae32-66e3725e4335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:08.007944 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-5f8a49ef-e64b-402d-ae32-66e3725e4335 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:08.007944 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:08.007944 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:08.007944 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:08.007944 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:08.007944 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:08.007944 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:08.007944 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:08.007944 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:08.007944 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:08.007944 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:08.007944 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:08.007944 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:08.007944 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:08.007944 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:08.007944 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:08.008891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:08.008891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:08.008891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:08.008891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:08.008891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:08.008891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:08.008891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:08.008891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:08.008891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:08.008891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:08.008891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:08.008891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:08.008891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:08.008891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:08.008891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:08.008891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:08.010249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:08.010249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:08.010249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:08.010249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:08.010249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:08.010249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:08.010249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:08.010249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:08.010249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:08.010249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:08.010249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:08.010249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:08.010249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 61/118] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:08.010249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a49ef-e64b-402d-ae32-66e3725e4335 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:08.018093 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:08.120324 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 58/119] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:55:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 635 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 22 23:55:08.128535 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ba3de-b52a-407b-8a1d-b373d2ffbcfb None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:08.184204 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a49ef-e64b-402d-ae32-66e3725e4335 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:08.234332 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af50d0-d615-425d-ab0d-990e241c593a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:08.236192 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af50d0-d615-425d-ab0d-990e241c593a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:08.237564 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af50d0-d615-425d-ab0d-990e241c593a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:08.240508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af50d0-d615-425d-ab0d-990e241c593a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:08.330986 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af50d0-d615-425d-ab0d-990e241c593a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:08.331424 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:08.332456 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af50d0-d615-425d-ab0d-990e241c593a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:08.334534 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:08.340895 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ba3de-b52a-407b-8a1d-b373d2ffbcfb None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:08.380662 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fcb428-9407-49fb-9ea2-714129c52b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:08.381377 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcb428-9407-49fb-9ea2-714129c52b4e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:08.381669 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcb428-9407-49fb-9ea2-714129c52b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:08.381952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fcb428-9407-49fb-9ea2-714129c52b4e None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:08.382725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fcb428-9407-49fb-9ea2-714129c52b4e None admin] RBAC: Adding query filter params (name=member) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:08.383038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fcb428-9407-49fb-9ea2-714129c52b4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:08.384411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fcb428-9407-49fb-9ea2-714129c52b4e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:08.393815 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 59/120] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Oct 22 23:55:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:08.394657 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fcb428-9407-49fb-9ea2-714129c52b4e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:08.457921 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 62/121] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:55:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 22 23:55:08.458532 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af50d0-d615-425d-ab0d-990e241c593a None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:08.535147 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fcb428-9407-49fb-9ea2-714129c52b4e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:08.570593 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5d2b3-81a2-4769-9740-9f5e342b1d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:08.571224 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5d2b3-81a2-4769-9740-9f5e342b1d1e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:08.571512 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5d2b3-81a2-4769-9740-9f5e342b1d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:08.571789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5d2b3-81a2-4769-9740-9f5e342b1d1e None admin] PATH_INFO: `/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:08.573017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5d2b3-81a2-4769-9740-9f5e342b1d1e None admin] RBAC: Authorizing `identity:get_project(project_id=4d6b66c470f74a4dac6f66e6fcee2a51)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:08.574535 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5d2b3-81a2-4769-9740-9f5e342b1d1e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:08.576396 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 60/122] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:55:08 2022] GET /identity/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:08.577082 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5d2b3-81a2-4769-9740-9f5e342b1d1e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:08.582624 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af50d0-d615-425d-ab0d-990e241c593a None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:08.584157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:08.604863 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:08.624816 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fdc340-10ce-499c-a952-be208637ede6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:08.625995 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fdc340-10ce-499c-a952-be208637ede6 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:08.626282 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fdc340-10ce-499c-a952-be208637ede6 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:08.626566 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fdc340-10ce-499c-a952-be208637ede6 None admin] PATH_INFO: `/v3/users/214a39c2cb9344d8bab53fdc6b5324b0` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:08.627773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fdc340-10ce-499c-a952-be208637ede6 None admin] RBAC: Authorizing `identity:get_user(user_id=214a39c2cb9344d8bab53fdc6b5324b0)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:08.629291 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fdc340-10ce-499c-a952-be208637ede6 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:08.631786 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 63/123] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:55:08 2022] GET /identity/v3/users/214a39c2cb9344d8bab53fdc6b5324b0 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:08.632613 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fdc340-10ce-499c-a952-be208637ede6 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:08.692831 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5d2b3-81a2-4769-9740-9f5e342b1d1e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:08.727865 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc99d5-0df6-4d1d-8338-3b4d91753ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:08.728427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc99d5-0df6-4d1d-8338-3b4d91753ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:08.728755 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc99d5-0df6-4d1d-8338-3b4d91753ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:08.729091 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc99d5-0df6-4d1d-8338-3b4d91753ca8 None admin] PATH_INFO: `/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:08.730258 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc99d5-0df6-4d1d-8338-3b4d91753ca8 None admin] RBAC: Authorizing `identity:get_project(project_id=4d6b66c470f74a4dac6f66e6fcee2a51)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:08.731747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcc99d5-0df6-4d1d-8338-3b4d91753ca8 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:08.733403 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 61/124] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:55:08 2022] GET /identity/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:08.734171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc99d5-0df6-4d1d-8338-3b4d91753ca8 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:08.758650 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fdc340-10ce-499c-a952-be208637ede6 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:08.797566 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e09bb-be07-4e83-b923-36be5cc2edcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:08.798701 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e09bb-be07-4e83-b923-36be5cc2edcb None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:08.799133 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e09bb-be07-4e83-b923-36be5cc2edcb None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:08.799535 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e09bb-be07-4e83-b923-36be5cc2edcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:08.800319 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e09bb-be07-4e83-b923-36be5cc2edcb None admin] RBAC: Adding query filter params (role.id=839a55263e2c4cbaa2f945b75cf298d6, scope.project.id=4d6b66c470f74a4dac6f66e6fcee2a51, user.id=ac8d80b2c4a24dfea25c6de809bc37e3) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:08.800692 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e09bb-be07-4e83-b923-36be5cc2edcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:08.802202 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e09bb-be07-4e83-b923-36be5cc2edcb None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:08.808537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 64/125] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:55:08 2022] GET /identity/v3/role_assignments?user.id=ac8d80b2c4a24dfea25c6de809bc37e3&scope.project.id=4d6b66c470f74a4dac6f66e6fcee2a51&role.id=839a55263e2c4cbaa2f945b75cf298d6 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:08.809970 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e09bb-be07-4e83-b923-36be5cc2edcb None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:08.853351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:08.860477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:08.873970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc99d5-0df6-4d1d-8338-3b4d91753ca8 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:08.949651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e09bb-be07-4e83-b923-36be5cc2edcb None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:08.954033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd89eae-051f-4616-b88d-f092f5fe8bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:08.955033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd89eae-051f-4616-b88d-f092f5fe8bdd None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:08.955533 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd89eae-051f-4616-b88d-f092f5fe8bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:08.956014 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd89eae-051f-4616-b88d-f092f5fe8bdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:09.025724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933d3c0-fd2a-4553-8c45-24ea66a97f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:09.027038 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933d3c0-fd2a-4553-8c45-24ea66a97f9f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:09.027909 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933d3c0-fd2a-4553-8c45-24ea66a97f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:09.030412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933d3c0-fd2a-4553-8c45-24ea66a97f9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:09.032166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933d3c0-fd2a-4553-8c45-24ea66a97f9f None admin] RBAC: Adding query filter params (role.id=cf8116e4e82748ac872d8763d113ba61, scope.project.id=4d6b66c470f74a4dac6f66e6fcee2a51, user.id=214a39c2cb9344d8bab53fdc6b5324b0) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:09.032504 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933d3c0-fd2a-4553-8c45-24ea66a97f9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:09.033999 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933d3c0-fd2a-4553-8c45-24ea66a97f9f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:09.041479 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 65/126] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:55:08 2022] GET /identity/v3/role_assignments?user.id=214a39c2cb9344d8bab53fdc6b5324b0&scope.project.id=4d6b66c470f74a4dac6f66e6fcee2a51&role.id=cf8116e4e82748ac872d8763d113ba61 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:09.075424 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd89eae-051f-4616-b88d-f092f5fe8bdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:09.077334 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd89eae-051f-4616-b88d-f092f5fe8bdd None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:09.185314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:09.197808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 62/127] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:55:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 22 23:55:09.205236 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933d3c0-fd2a-4553-8c45-24ea66a97f9f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:09.325279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:09.329174 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933d3c0-fd2a-4553-8c45-24ea66a97f9f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:09.366466 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e9fbb-66a5-4184-8d41-86ede9d08380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:09.367105 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e9fbb-66a5-4184-8d41-86ede9d08380 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:09.367398 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e9fbb-66a5-4184-8d41-86ede9d08380 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:09.367684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e9fbb-66a5-4184-8d41-86ede9d08380 None admin] PATH_INFO: `/v3/projects/aa6452edd45c426e9eb7e0a891ba2952` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:09.368848 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e9fbb-66a5-4184-8d41-86ede9d08380 None admin] RBAC: Authorizing `identity:get_project(project_id=aa6452edd45c426e9eb7e0a891ba2952)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:09.370310 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e9fbb-66a5-4184-8d41-86ede9d08380 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:09.371959 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 66/128] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:55:09 2022] GET /identity/v3/projects/aa6452edd45c426e9eb7e0a891ba2952 => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:09.378691 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd89eae-051f-4616-b88d-f092f5fe8bdd None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:09.481459 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd89eae-051f-4616-b88d-f092f5fe8bdd None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:09.517024 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a646731-e6fb-4488-99c2-f6fc44a76876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:09.517516 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a646731-e6fb-4488-99c2-f6fc44a76876 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:09.517802 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a646731-e6fb-4488-99c2-f6fc44a76876 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:09.518079 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a646731-e6fb-4488-99c2-f6fc44a76876 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:09.518805 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a646731-e6fb-4488-99c2-f6fc44a76876 None admin] RBAC: Adding query filter params (role.id=4bd2874c95634a50aac9fccbddc7c674, scope.project.id=aa6452edd45c426e9eb7e0a891ba2952, user.id=214a39c2cb9344d8bab53fdc6b5324b0) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:09.519115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a646731-e6fb-4488-99c2-f6fc44a76876 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:09.520624 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a646731-e6fb-4488-99c2-f6fc44a76876 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:09.532537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 63/129] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:55:09 2022] GET /identity/v3/role_assignments?user.id=214a39c2cb9344d8bab53fdc6b5324b0&scope.project.id=aa6452edd45c426e9eb7e0a891ba2952&role.id=4bd2874c95634a50aac9fccbddc7c674 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:09.836458 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e71e8-c497-4b85-b1ca-beb5bf3c212d None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:09.836786 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e71e8-c497-4b85-b1ca-beb5bf3c212d None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:09.837069 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e71e8-c497-4b85-b1ca-beb5bf3c212d None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:09.838428 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 67/130] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:09.844260 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91295889-c1a7-4dfe-a140-136a4f3664c0 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:09.844577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91295889-c1a7-4dfe-a140-136a4f3664c0 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:09.844862 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91295889-c1a7-4dfe-a140-136a4f3664c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:09.864136 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-91295889-c1a7-4dfe-a140-136a4f3664c0 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:09.865405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-91295889-c1a7-4dfe-a140-136a4f3664c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:09.916179 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:09.933362 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 64/131] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:09.964764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfb7ea-1da4-4551-a833-2615502c62a0 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:09.965064 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfb7ea-1da4-4551-a833-2615502c62a0 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:09.965349 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfb7ea-1da4-4551-a833-2615502c62a0 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:09.966700 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 68/132] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:09.972023 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16ce30-4234-4a7c-b042-35f1abe27576 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:09.972337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16ce30-4234-4a7c-b042-35f1abe27576 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:09.972649 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16ce30-4234-4a7c-b042-35f1abe27576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:09.990178 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-6f16ce30-4234-4a7c-b042-35f1abe27576 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:09.991341 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-6f16ce30-4234-4a7c-b042-35f1abe27576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:10.057239 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 65/133] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:10.202183 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfb7ea-1da4-4551-a833-2615502c62a0 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:10.304773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfb7ea-1da4-4551-a833-2615502c62a0 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:10.324644 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16ce30-4234-4a7c-b042-35f1abe27576 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:10.339558 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcaa54-4aa3-4340-ab54-8298ac28d6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:10.340193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcaa54-4aa3-4340-ab54-8298ac28d6f4 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:10.340512 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcaa54-4aa3-4340-ab54-8298ac28d6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:10.340830 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fcaa54-4aa3-4340-ab54-8298ac28d6f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:10.341938 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fcaa54-4aa3-4340-ab54-8298ac28d6f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:10.343390 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fcaa54-4aa3-4340-ab54-8298ac28d6f4 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:10.345555 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 69/134] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:10.350946 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcaa54-4aa3-4340-ab54-8298ac28d6f4 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:10.384782 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:10.424590 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16ce30-4234-4a7c-b042-35f1abe27576 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:10.441099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:10.450324 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fcaa54-4aa3-4340-ab54-8298ac28d6f4 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:10.460273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e84a63-a907-49d6-a83a-90d750774044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:10.460952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e84a63-a907-49d6-a83a-90d750774044 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:10.461242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e84a63-a907-49d6-a83a-90d750774044 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:10.461520 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e84a63-a907-49d6-a83a-90d750774044 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:10.471608 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e84a63-a907-49d6-a83a-90d750774044 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:10.476433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-28e84a63-a907-49d6-a83a-90d750774044 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 22 23:55:10.476433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:10.476433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:10.476433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:10.476433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:10.476433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:10.476433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:10.476433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:10.476433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:10.476433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:10.476433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:10.476433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:10.476433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:10.476433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:10.476433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:10.476433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:10.477368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:10.477368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:10.477368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:10.477368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:10.477368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:10.477368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:10.477368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:10.477368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:10.477368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:10.477368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:10.477368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:10.477368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:10.477368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:10.477368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:10.477368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:10.477368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:10.478153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:10.478153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:10.478153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:10.478153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:10.478153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:10.478153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:10.478153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:10.478153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:10.478153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:10.478153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:10.478153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 22 23:55:10.478153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:10.478153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 66/135] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:55:10.483407 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e84a63-a907-49d6-a83a-90d750774044 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:10.483627 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab3443-0394-488c-abdb-a5decca8d3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:10.484234 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab3443-0394-488c-abdb-a5decca8d3c4 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:10.485049 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab3443-0394-488c-abdb-a5decca8d3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:10.485694 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab3443-0394-488c-abdb-a5decca8d3c4 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:10.486346 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab3443-0394-488c-abdb-a5decca8d3c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:10.487778 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab3443-0394-488c-abdb-a5decca8d3c4 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:10.508185 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 70/136] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Oct 22 23:55:10 2022] POST /identity/v3/users => generated 283 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 22 23:55:10.590205 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e84a63-a907-49d6-a83a-90d750774044 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:10.624754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ceb8e2-fde2-4bf3-8594-be78e52e06b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:10.626079 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ceb8e2-fde2-4bf3-8594-be78e52e06b7 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:10.626509 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ceb8e2-fde2-4bf3-8594-be78e52e06b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:10.626797 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ceb8e2-fde2-4bf3-8594-be78e52e06b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:10.627472 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ceb8e2-fde2-4bf3-8594-be78e52e06b7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:10.627779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ceb8e2-fde2-4bf3-8594-be78e52e06b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:10.629223 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ceb8e2-fde2-4bf3-8594-be78e52e06b7 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:10.638677 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 67/137] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Oct 22 23:55:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:10.644879 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab3443-0394-488c-abdb-a5decca8d3c4 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:10.743983 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:10.755153 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab3443-0394-488c-abdb-a5decca8d3c4 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:10.792540 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d03fbb-3087-4a08-a327-a3f55fdf4102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:10.793232 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d03fbb-3087-4a08-a327-a3f55fdf4102 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:10.793523 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d03fbb-3087-4a08-a327-a3f55fdf4102 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:10.793806 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d03fbb-3087-4a08-a327-a3f55fdf4102 None admin] PATH_INFO: `/v3/users/4a0fb7285e844f7a8d0e53f7b6814786` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:10.794964 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d03fbb-3087-4a08-a327-a3f55fdf4102 None admin] RBAC: Authorizing `identity:get_user(user_id=4a0fb7285e844f7a8d0e53f7b6814786)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:10.796397 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d03fbb-3087-4a08-a327-a3f55fdf4102 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:10.799191 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 71/138] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:55:10 2022] GET /identity/v3/users/4a0fb7285e844f7a8d0e53f7b6814786 => generated 289 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:10.804931 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ceb8e2-fde2-4bf3-8594-be78e52e06b7 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:10.850603 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:10.918688 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ceb8e2-fde2-4bf3-8594-be78e52e06b7 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:10.957633 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab41b95-71e8-468a-b322-c7ecf2a56a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:10.958262 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab41b95-71e8-468a-b322-c7ecf2a56a20 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:10.958543 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab41b95-71e8-468a-b322-c7ecf2a56a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:10.958822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab41b95-71e8-468a-b322-c7ecf2a56a20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:11.030029 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab41b95-71e8-468a-b322-c7ecf2a56a20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:11.031546 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab41b95-71e8-468a-b322-c7ecf2a56a20 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:11.073101 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:11.143467 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 68/139] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:55:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 22 23:55:11.149408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d03fbb-3087-4a08-a327-a3f55fdf4102 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:11.259003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d03fbb-3087-4a08-a327-a3f55fdf4102 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:11.295398 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a420f4e6-227b-4b8e-9c0d-4b119049c1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:11.296372 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420f4e6-227b-4b8e-9c0d-4b119049c1c8 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:11.296912 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420f4e6-227b-4b8e-9c0d-4b119049c1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:11.297932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420f4e6-227b-4b8e-9c0d-4b119049c1c8 None admin] PATH_INFO: `/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:11.299290 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a420f4e6-227b-4b8e-9c0d-4b119049c1c8 None admin] RBAC: Authorizing `identity:get_project(project_id=4d6b66c470f74a4dac6f66e6fcee2a51)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:11.300976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a420f4e6-227b-4b8e-9c0d-4b119049c1c8 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:11.302829 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 72/140] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:55:11 2022] GET /identity/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:11.312519 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab41b95-71e8-468a-b322-c7ecf2a56a20 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:11.419126 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab41b95-71e8-468a-b322-c7ecf2a56a20 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:11.454680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88578649-fefe-4794-8646-cce4fca4c13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:11.455292 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88578649-fefe-4794-8646-cce4fca4c13a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:11.455579 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88578649-fefe-4794-8646-cce4fca4c13a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:11.455859 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88578649-fefe-4794-8646-cce4fca4c13a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:11.456607 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88578649-fefe-4794-8646-cce4fca4c13a None admin] RBAC: Adding query filter params (role.id=159c6380cd984e998bb6a97ed7d98dac, scope.project.id=4d6b66c470f74a4dac6f66e6fcee2a51, user.id=4a0fb7285e844f7a8d0e53f7b6814786) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:11.456924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88578649-fefe-4794-8646-cce4fca4c13a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:11.458379 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88578649-fefe-4794-8646-cce4fca4c13a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:11.464679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 69/141] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:55:11 2022] GET /identity/v3/role_assignments?user.id=4a0fb7285e844f7a8d0e53f7b6814786&scope.project.id=4d6b66c470f74a4dac6f66e6fcee2a51&role.id=159c6380cd984e998bb6a97ed7d98dac => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:12.796356 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8a86e-e3e1-4e76-87ec-6662406e0aed None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:12.796900 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8a86e-e3e1-4e76-87ec-6662406e0aed None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:12.796975 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8a86e-e3e1-4e76-87ec-6662406e0aed None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:12.798919 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 73/142] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:12.804104 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbb861-51a7-4689-8e89-1e993b068772 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:12.804545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbb861-51a7-4689-8e89-1e993b068772 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:12.804709 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbb861-51a7-4689-8e89-1e993b068772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:12.810981 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:12.823621 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-4dfbb861-51a7-4689-8e89-1e993b068772 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:12.824855 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-4dfbb861-51a7-4689-8e89-1e993b068772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:12.896606 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 70/143] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:12.988202 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2f170-e8c8-4849-bc49-4392e57bc857 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:12.988531 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2f170-e8c8-4849-bc49-4392e57bc857 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:12.988813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2f170-e8c8-4849-bc49-4392e57bc857 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:12.990182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 74/144] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:12.995475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46bbd1e-26d5-474a-accb-99a4bea1a40e None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:12.995769 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46bbd1e-26d5-474a-accb-99a4bea1a40e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:12.996049 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46bbd1e-26d5-474a-accb-99a4bea1a40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:13.013995 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-b46bbd1e-26d5-474a-accb-99a4bea1a40e None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:13.015175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b46bbd1e-26d5-474a-accb-99a4bea1a40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:13.038171 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddfb2b-c014-4e82-a88c-f3d94fd6b185 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:13.038545 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddfb2b-c014-4e82-a88c-f3d94fd6b185 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:13.038893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eddfb2b-c014-4e82-a88c-f3d94fd6b185 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:13.040890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 75/145] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:13.045881 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc5c1f-1550-4550-94d4-bff5c4acd9fc None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:13.046274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc5c1f-1550-4550-94d4-bff5c4acd9fc None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:13.046702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc5c1f-1550-4550-94d4-bff5c4acd9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:13.066624 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-10fc5c1f-1550-4550-94d4-bff5c4acd9fc None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:13.067820 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-10fc5c1f-1550-4550-94d4-bff5c4acd9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:13.085685 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 71/146] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:13.090418 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:13.146709 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 76/147] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 22 23:55:13.170378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46bbd1e-26d5-474a-accb-99a4bea1a40e None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:13.232417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:13.277262 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46bbd1e-26d5-474a-accb-99a4bea1a40e None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:13.311297 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f6b37-d67d-412d-a499-e0b11aa6bccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:13.311855 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f6b37-d67d-412d-a499-e0b11aa6bccc None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:13.312136 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f6b37-d67d-412d-a499-e0b11aa6bccc None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:13.312408 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f6b37-d67d-412d-a499-e0b11aa6bccc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:13.323001 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f6b37-d67d-412d-a499-e0b11aa6bccc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:13.327814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-213f6b37-d67d-412d-a499-e0b11aa6bccc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:55:13.327814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:13.327814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:13.327814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:13.327814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:13.327814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:13.327814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:13.327814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:13.327814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:13.327814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:13.327814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:13.327814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:13.327814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:13.327814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:13.327814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:13.327814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:13.329009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:13.329009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:13.329009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:13.329009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:13.329009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:13.329009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:13.329009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:13.329009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:13.329009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:13.329009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:13.329009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:13.329009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:13.329009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:13.329009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:13.329009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:13.329009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:13.329960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:13.329960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:13.329960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:13.329960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:13.329960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:13.329960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:13.329960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:13.329960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:13.329960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:13.329960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:13.329960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:55:13.329960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:13.329960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 72/148] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Oct 22 23:55:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:13.336658 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc5c1f-1550-4550-94d4-bff5c4acd9fc None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:13.354211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f6b37-d67d-412d-a499-e0b11aa6bccc None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:13.443356 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc5c1f-1550-4550-94d4-bff5c4acd9fc None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:13.461454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f6b37-d67d-412d-a499-e0b11aa6bccc None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:13.479870 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d16ac-fd6b-4b98-8933-05feecbccf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:13.480473 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d16ac-fd6b-4b98-8933-05feecbccf28 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:13.480788 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d16ac-fd6b-4b98-8933-05feecbccf28 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:13.481065 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d16ac-fd6b-4b98-8933-05feecbccf28 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:13.481721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d16ac-fd6b-4b98-8933-05feecbccf28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:13.482026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d16ac-fd6b-4b98-8933-05feecbccf28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:13.483392 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d16ac-fd6b-4b98-8933-05feecbccf28 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:13.484525 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:13.492605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 77/149] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:13.493483 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d16ac-fd6b-4b98-8933-05feecbccf28 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:13.501355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc87d26-ae59-40d3-a92f-dfce23b462a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:13.502081 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc87d26-ae59-40d3-a92f-dfce23b462a8 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:13.502364 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc87d26-ae59-40d3-a92f-dfce23b462a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:13.502640 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc87d26-ae59-40d3-a92f-dfce23b462a8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:13.511947 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:13.513490 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc87d26-ae59-40d3-a92f-dfce23b462a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:13.518982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-cbc87d26-ae59-40d3-a92f-dfce23b462a8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 22 23:55:13.518982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:13.518982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:13.518982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:13.518982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:13.518982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:13.518982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:13.518982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:13.518982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:13.518982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:13.518982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:13.518982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:13.518982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:13.518982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:13.518982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:13.518982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:13.520034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:13.520034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:13.520034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:13.520034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:13.520034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:13.520034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:13.520034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:13.520034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:13.520034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:13.520034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:13.520034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:13.520034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:13.520034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:13.520034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:13.520034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:13.520034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:13.520875 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:13.520875 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:13.520875 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:13.520875 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:13.520875 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:13.520875 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:13.520875 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:13.520875 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:13.520875 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:13.520875 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:13.520875 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 22 23:55:13.520875 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:13.520875 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 73/150] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Oct 22 23:55:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:55:13.520875 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc87d26-ae59-40d3-a92f-dfce23b462a8 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:13.597073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d16ac-fd6b-4b98-8933-05feecbccf28 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:13.620920 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc87d26-ae59-40d3-a92f-dfce23b462a8 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:13.629893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5534e-2f3d-4856-8350-3b96035fd383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:13.630517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5534e-2f3d-4856-8350-3b96035fd383 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:13.630807 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5534e-2f3d-4856-8350-3b96035fd383 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:13.631081 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b5534e-2f3d-4856-8350-3b96035fd383 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:13.632181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5534e-2f3d-4856-8350-3b96035fd383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:13.633653 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b5534e-2f3d-4856-8350-3b96035fd383 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:13.635225 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 78/151] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:13.635991 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5534e-2f3d-4856-8350-3b96035fd383 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:13.657606 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4ff26-a7d6-4345-ac25-e02132672144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:13.658182 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4ff26-a7d6-4345-ac25-e02132672144 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:13.658464 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4ff26-a7d6-4345-ac25-e02132672144 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:13.658758 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4ff26-a7d6-4345-ac25-e02132672144 None admin] PATH_INFO: `/v3/users/2f0267a7d5d84439880c6a9ca38dd660` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:13.675038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce4ff26-a7d6-4345-ac25-e02132672144 None admin] RBAC: Authorizing `identity:get_user(user_id=2f0267a7d5d84439880c6a9ca38dd660)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:13.676820 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce4ff26-a7d6-4345-ac25-e02132672144 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:13.678646 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 74/152] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:55:13 2022] GET /identity/v3/users/2f0267a7d5d84439880c6a9ca38dd660 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:13.679542 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4ff26-a7d6-4345-ac25-e02132672144 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:13.721573 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:13.727510 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:13.742376 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b5534e-2f3d-4856-8350-3b96035fd383 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:13.775535 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a66d4-af95-43f6-9e05-91093e11c116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:13.776140 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a66d4-af95-43f6-9e05-91093e11c116 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:13.776424 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a66d4-af95-43f6-9e05-91093e11c116 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:13.776735 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a66d4-af95-43f6-9e05-91093e11c116 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:13.777390 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a66d4-af95-43f6-9e05-91093e11c116 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:13.777694 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a66d4-af95-43f6-9e05-91093e11c116 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:13.779050 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a66d4-af95-43f6-9e05-91093e11c116 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:13.779220 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4ff26-a7d6-4345-ac25-e02132672144 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:13.787368 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 79/153] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:55:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:13.792248 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a66d4-af95-43f6-9e05-91093e11c116 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:13.813800 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59043baa-fb13-4e36-820d-87aed76afd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:13.814402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59043baa-fb13-4e36-820d-87aed76afd50 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:13.814685 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59043baa-fb13-4e36-820d-87aed76afd50 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:13.814954 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59043baa-fb13-4e36-820d-87aed76afd50 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:13.815854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59043baa-fb13-4e36-820d-87aed76afd50 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:13.817307 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59043baa-fb13-4e36-820d-87aed76afd50 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:13.838056 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 75/154] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Oct 22 23:55:13 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:55:13.838056 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59043baa-fb13-4e36-820d-87aed76afd50 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:13.902687 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a66d4-af95-43f6-9e05-91093e11c116 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:13.944357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d221f-9c76-42b2-84d5-d483e0122acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:13.945040 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d221f-9c76-42b2-84d5-d483e0122acd None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:13.945325 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d221f-9c76-42b2-84d5-d483e0122acd None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:13.945605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d221f-9c76-42b2-84d5-d483e0122acd None admin] PATH_INFO: `/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:13.946754 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d221f-9c76-42b2-84d5-d483e0122acd None admin] RBAC: Authorizing `identity:get_project(project_id=79baed4da64f4fc2b256aa6fc529fbd2)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:13.948243 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d221f-9c76-42b2-84d5-d483e0122acd None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:13.948855 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:13.950374 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 80/155] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:55:13 2022] GET /identity/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:13.957189 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d221f-9c76-42b2-84d5-d483e0122acd None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:13.961614 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59043baa-fb13-4e36-820d-87aed76afd50 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:13.977414 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:13.998066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f42f60-e7ba-4535-9034-088980ae1f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:13.998746 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f42f60-e7ba-4535-9034-088980ae1f5e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:13.999028 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f42f60-e7ba-4535-9034-088980ae1f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:13.999305 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f42f60-e7ba-4535-9034-088980ae1f5e None admin] PATH_INFO: `/v3/users/2f0267a7d5d84439880c6a9ca38dd660` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:14.001067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f42f60-e7ba-4535-9034-088980ae1f5e None admin] RBAC: Authorizing `identity:get_user(user_id=2f0267a7d5d84439880c6a9ca38dd660)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:14.002528 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f42f60-e7ba-4535-9034-088980ae1f5e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:14.004181 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 76/156] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:55:13 2022] GET /identity/v3/users/2f0267a7d5d84439880c6a9ca38dd660 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:14.009651 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f42f60-e7ba-4535-9034-088980ae1f5e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:14.077799 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d221f-9c76-42b2-84d5-d483e0122acd None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:14.121782 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1884310-1da1-4019-a9b6-31b0f1eb3d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:14.121782 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1884310-1da1-4019-a9b6-31b0f1eb3d0b None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:14.121782 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1884310-1da1-4019-a9b6-31b0f1eb3d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:14.122084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1884310-1da1-4019-a9b6-31b0f1eb3d0b None admin] PATH_INFO: `/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2/users/2f0267a7d5d84439880c6a9ca38dd660/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:14.124080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1884310-1da1-4019-a9b6-31b0f1eb3d0b None admin] RBAC: Authorizing `identity:create_grant(project_id=79baed4da64f4fc2b256aa6fc529fbd2, user_id=2f0267a7d5d84439880c6a9ca38dd660, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:14.126046 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1884310-1da1-4019-a9b6-31b0f1eb3d0b None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:14.138353 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f42f60-e7ba-4535-9034-088980ae1f5e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:14.143514 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 81/157] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:55:13 2022] PUT /identity/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2/users/2f0267a7d5d84439880c6a9ca38dd660/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:55:14.181150 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5c18b-2412-4515-9f1d-5120c72963b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:14.182216 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5c18b-2412-4515-9f1d-5120c72963b2 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:14.182706 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5c18b-2412-4515-9f1d-5120c72963b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:14.183187 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5c18b-2412-4515-9f1d-5120c72963b2 None admin] PATH_INFO: `/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:14.184469 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5c18b-2412-4515-9f1d-5120c72963b2 None admin] RBAC: Authorizing `identity:get_project(project_id=79baed4da64f4fc2b256aa6fc529fbd2)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:14.186021 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5c18b-2412-4515-9f1d-5120c72963b2 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:14.188765 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 77/158] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:55:14 2022] GET /identity/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:55:14.196577 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1884310-1da1-4019-a9b6-31b0f1eb3d0b None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:14.286115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:14.312351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1884310-1da1-4019-a9b6-31b0f1eb3d0b None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:14.349310 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2821873-1b24-4d23-a23a-902de1ca37dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:14.349980 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2821873-1b24-4d23-a23a-902de1ca37dd None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:14.350845 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2821873-1b24-4d23-a23a-902de1ca37dd None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:14.351136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2821873-1b24-4d23-a23a-902de1ca37dd None admin] PATH_INFO: `/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2/users/2f0267a7d5d84439880c6a9ca38dd660/roles/cf8116e4e82748ac872d8763d113ba61` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:14.353360 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2821873-1b24-4d23-a23a-902de1ca37dd None admin] RBAC: Authorizing `identity:create_grant(project_id=79baed4da64f4fc2b256aa6fc529fbd2, user_id=2f0267a7d5d84439880c6a9ca38dd660, role_id=cf8116e4e82748ac872d8763d113ba61)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:14.354807 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2821873-1b24-4d23-a23a-902de1ca37dd None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:14.370825 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 82/159] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:55:14 2022] PUT /identity/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2/users/2f0267a7d5d84439880c6a9ca38dd660/roles/cf8116e4e82748ac872d8763d113ba61 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:55:16.177373 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a370e7b-6a45-4bd1-ab3f-e1f776620576 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:16.177914 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a370e7b-6a45-4bd1-ab3f-e1f776620576 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:16.177914 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a370e7b-6a45-4bd1-ab3f-e1f776620576 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:16.179260 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 78/160] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:16.184730 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d60877-df5c-48be-8ac3-b341d4f43985 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:16.185569 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d60877-df5c-48be-8ac3-b341d4f43985 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:16.186046 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d60877-df5c-48be-8ac3-b341d4f43985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:16.206205 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-35d60877-df5c-48be-8ac3-b341d4f43985 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:16.207705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-35d60877-df5c-48be-8ac3-b341d4f43985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:16.268005 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a6b31-0986-4236-a8fc-1e55735775e8 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:16.268336 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a6b31-0986-4236-a8fc-1e55735775e8 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:16.268580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a6b31-0986-4236-a8fc-1e55735775e8 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:16.270083 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 79/161] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:16.275473 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19222dca-d037-4401-92aa-0e4ac930070f None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:16.276049 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19222dca-d037-4401-92aa-0e4ac930070f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:16.276566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19222dca-d037-4401-92aa-0e4ac930070f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:16.287596 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 83/162] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:16.295379 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-19222dca-d037-4401-92aa-0e4ac930070f None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:16.296841 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-19222dca-d037-4401-92aa-0e4ac930070f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:16.325192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:16.368938 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 80/163] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 22 23:55:16.374146 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3088f-f45e-462d-8f73-659d056b705d None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:16.374710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3088f-f45e-462d-8f73-659d056b705d None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:16.375188 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3088f-f45e-462d-8f73-659d056b705d None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:16.377086 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 84/164] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:16.382518 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6326b-9f32-412c-a869-e04952a12bfd None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:16.382807 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6326b-9f32-412c-a869-e04952a12bfd None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:16.383083 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6326b-9f32-412c-a869-e04952a12bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:16.401901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-4ad6326b-9f32-412c-a869-e04952a12bfd None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:16.403068 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-4ad6326b-9f32-412c-a869-e04952a12bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:16.475469 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 81/165] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 22 23:55:16.578449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb9c5c-3ce1-4a16-88d5-596e1ef46826 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:16.578449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb9c5c-3ce1-4a16-88d5-596e1ef46826 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:16.578449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cb9c5c-3ce1-4a16-88d5-596e1ef46826 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:16.578449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 85/166] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:16.584226 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83aba74-9329-49bc-aa83-20b64925b4dd None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:16.584766 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83aba74-9329-49bc-aa83-20b64925b4dd None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:16.585241 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83aba74-9329-49bc-aa83-20b64925b4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:16.590804 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb9c5c-3ce1-4a16-88d5-596e1ef46826 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:16.607540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-a83aba74-9329-49bc-aa83-20b64925b4dd None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:16.608919 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a83aba74-9329-49bc-aa83-20b64925b4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:16.671974 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:16.689178 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 82/167] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:16.689860 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83aba74-9329-49bc-aa83-20b64925b4dd None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:16.717005 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:16.731836 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cb9c5c-3ce1-4a16-88d5-596e1ef46826 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:16.772003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ca1fa-4514-4495-880a-d1e69dc69af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:16.772781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ca1fa-4514-4495-880a-d1e69dc69af8 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:16.773065 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ca1fa-4514-4495-880a-d1e69dc69af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:16.773342 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ca1fa-4514-4495-880a-d1e69dc69af8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:16.784708 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ca1fa-4514-4495-880a-d1e69dc69af8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:16.789588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-6e4ca1fa-4514-4495-880a-d1e69dc69af8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:16.789588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:16.789588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:16.789588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:16.789588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:16.789588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:16.789588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:16.789588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:16.789588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:16.789588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:16.789588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:16.789588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:16.789588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:16.789588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:16.789588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:16.789588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:16.790568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:16.790568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:16.790568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:16.790568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:16.790568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:16.790568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:16.790568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:16.790568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:16.790568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:16.790568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:16.790568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:16.790568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:16.790568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:16.790568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:16.790568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:16.790568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:16.791366 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:16.791366 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:16.791366 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:16.791366 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:16.791366 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:16.791366 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:16.791366 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:16.791366 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:16.791366 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:16.791366 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:16.791366 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:16.791366 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:16.791366 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 86/168] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:55:16.792039 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ca1fa-4514-4495-880a-d1e69dc69af8 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:16.808936 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83aba74-9329-49bc-aa83-20b64925b4dd None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:16.846059 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e49c7-fa9f-4f11-8497-6a7995f83a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:16.846686 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e49c7-fa9f-4f11-8497-6a7995f83a75 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:16.846965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e49c7-fa9f-4f11-8497-6a7995f83a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:16.847241 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77e49c7-fa9f-4f11-8497-6a7995f83a75 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:16.848332 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e49c7-fa9f-4f11-8497-6a7995f83a75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:16.849841 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77e49c7-fa9f-4f11-8497-6a7995f83a75 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:16.851474 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 83/169] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:16.856660 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e49c7-fa9f-4f11-8497-6a7995f83a75 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:16.910233 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ca1fa-4514-4495-880a-d1e69dc69af8 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:16.938764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:16.950278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208a1c1-f50a-46b4-8c96-023b1feda6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:16.951111 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:16.951220 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208a1c1-f50a-46b4-8c96-023b1feda6e9 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:16.951430 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208a1c1-f50a-46b4-8c96-023b1feda6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:16.951716 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208a1c1-f50a-46b4-8c96-023b1feda6e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:16.962049 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77e49c7-fa9f-4f11-8497-6a7995f83a75 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:16.964919 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208a1c1-f50a-46b4-8c96-023b1feda6e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:16.970041 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-8208a1c1-f50a-46b4-8c96-023b1feda6e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:55:16.970041 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:16.970041 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:16.970041 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:16.970041 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:16.970041 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:16.970041 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:16.970041 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:16.970041 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:16.970041 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:16.970041 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:16.970041 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:16.970041 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:16.970041 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:16.970041 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:16.970041 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:16.971001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:16.971001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:16.971001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:16.971001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:16.971001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:16.971001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:16.971001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:16.971001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:16.971001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:16.971001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:16.971001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:16.971001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:16.971001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:16.971001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:16.971001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:16.971001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:16.971780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:16.971780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:16.971780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:16.971780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:16.971780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:16.971780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:16.971780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:16.971780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:16.971780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:16.971780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:16.971780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:55:16.971780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:16.971780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 87/170] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Oct 22 23:55:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:16.972433 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208a1c1-f50a-46b4-8c96-023b1feda6e9 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:16.997616 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1d4c4-d50a-4908-b7c0-52cdef7adc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:16.998230 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1d4c4-d50a-4908-b7c0-52cdef7adc68 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:16.998517 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1d4c4-d50a-4908-b7c0-52cdef7adc68 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:16.998800 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1d4c4-d50a-4908-b7c0-52cdef7adc68 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:16.999467 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1d4c4-d50a-4908-b7c0-52cdef7adc68 None admin] RBAC: Adding query filter params (name=member) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:16.999775 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1d4c4-d50a-4908-b7c0-52cdef7adc68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:17.001721 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1d4c4-d50a-4908-b7c0-52cdef7adc68 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:17.009949 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 84/171] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Oct 22 23:55:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:17.015229 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1d4c4-d50a-4908-b7c0-52cdef7adc68 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:17.072847 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208a1c1-f50a-46b4-8c96-023b1feda6e9 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:17.106836 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d68c6-9295-48c1-81b6-a5b656fd6b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:17.107646 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d68c6-9295-48c1-81b6-a5b656fd6b0d None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:17.108262 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d68c6-9295-48c1-81b6-a5b656fd6b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:17.108572 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d68c6-9295-48c1-81b6-a5b656fd6b0d None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:17.109435 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d68c6-9295-48c1-81b6-a5b656fd6b0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:17.111587 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d68c6-9295-48c1-81b6-a5b656fd6b0d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:17.114054 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1d4c4-d50a-4908-b7c0-52cdef7adc68 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:17.132450 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 88/172] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Oct 22 23:55:16 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:55:17.132450 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d68c6-9295-48c1-81b6-a5b656fd6b0d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:17.150535 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b3cc1-f3f3-44fd-9613-7eeb2d30de08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:17.151212 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b3cc1-f3f3-44fd-9613-7eeb2d30de08 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:17.151502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b3cc1-f3f3-44fd-9613-7eeb2d30de08 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:17.151782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b3cc1-f3f3-44fd-9613-7eeb2d30de08 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:17.163988 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b3cc1-f3f3-44fd-9613-7eeb2d30de08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:17.170017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-8c5b3cc1-f3f3-44fd-9613-7eeb2d30de08 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 22 23:55:17.170017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:17.170017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:17.170017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:17.170017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:17.170017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:17.170017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:17.170017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:17.170017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:17.170017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:17.170017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:17.170017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:17.170017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:17.170017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:17.170017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:17.170017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:17.171022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:17.171022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:17.171022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:17.171022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:17.171022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:17.171022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:17.171022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:17.171022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:17.171022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:17.171022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:17.171022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:17.171022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:17.171022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:17.171022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:17.171022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:17.171022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:17.173207 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:17.173207 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:17.173207 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:17.173207 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:17.173207 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:17.173207 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:17.173207 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:17.173207 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:17.173207 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:17.173207 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:17.173207 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 22 23:55:17.173207 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:17.173207 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 85/173] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Oct 22 23:55:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:17.173207 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b3cc1-f3f3-44fd-9613-7eeb2d30de08 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:17.190742 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:17.192479 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:17.259368 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d68c6-9295-48c1-81b6-a5b656fd6b0d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:17.287647 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b3cc1-f3f3-44fd-9613-7eeb2d30de08 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:17.298981 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483491e-4041-4ba1-95f5-c25d1bba829d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:17.299606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483491e-4041-4ba1-95f5-c25d1bba829d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:17.299895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483491e-4041-4ba1-95f5-c25d1bba829d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:17.300179 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483491e-4041-4ba1-95f5-c25d1bba829d None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:17.301152 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0483491e-4041-4ba1-95f5-c25d1bba829d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:17.301464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0483491e-4041-4ba1-95f5-c25d1bba829d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:17.303093 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0483491e-4041-4ba1-95f5-c25d1bba829d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:17.314584 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 89/174] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:17.315250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483491e-4041-4ba1-95f5-c25d1bba829d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:17.327684 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a8b17-b274-43d7-83d6-d58e1af29e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:17.327684 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a8b17-b274-43d7-83d6-d58e1af29e3c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:17.327684 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a8b17-b274-43d7-83d6-d58e1af29e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:17.327684 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a8b17-b274-43d7-83d6-d58e1af29e3c None admin] PATH_INFO: `/v3/users/ab9f668d87844bedb696a3318a64fab9` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:17.344081 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7a8b17-b274-43d7-83d6-d58e1af29e3c None admin] RBAC: Authorizing `identity:get_user(user_id=ab9f668d87844bedb696a3318a64fab9)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:17.345678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7a8b17-b274-43d7-83d6-d58e1af29e3c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:17.348161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 86/175] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:55:17 2022] GET /identity/v3/users/ab9f668d87844bedb696a3318a64fab9 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:17.348990 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a8b17-b274-43d7-83d6-d58e1af29e3c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:17.428894 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:17.446521 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483491e-4041-4ba1-95f5-c25d1bba829d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:17.450220 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:17.470557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a8b17-b274-43d7-83d6-d58e1af29e3c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:17.494640 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3dc124-0696-4cdb-8530-6f3b4af295eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:17.495258 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3dc124-0696-4cdb-8530-6f3b4af295eb None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:17.495540 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3dc124-0696-4cdb-8530-6f3b4af295eb None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:17.495815 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3dc124-0696-4cdb-8530-6f3b4af295eb None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:17.496501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3dc124-0696-4cdb-8530-6f3b4af295eb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:17.496848 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3dc124-0696-4cdb-8530-6f3b4af295eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:17.498235 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3dc124-0696-4cdb-8530-6f3b4af295eb None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:17.508100 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 90/176] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:55:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:17.508985 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3dc124-0696-4cdb-8530-6f3b4af295eb None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:17.517364 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b382285-c892-4f68-bd3f-e9baa9b18da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:17.518014 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b382285-c892-4f68-bd3f-e9baa9b18da6 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:17.518295 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b382285-c892-4f68-bd3f-e9baa9b18da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:17.518570 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b382285-c892-4f68-bd3f-e9baa9b18da6 None admin] PATH_INFO: `/v3/users/2f0267a7d5d84439880c6a9ca38dd660` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:17.519743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b382285-c892-4f68-bd3f-e9baa9b18da6 None admin] RBAC: Authorizing `identity:get_user(user_id=2f0267a7d5d84439880c6a9ca38dd660)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:17.521917 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b382285-c892-4f68-bd3f-e9baa9b18da6 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:17.524194 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 87/177] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:55:17 2022] GET /identity/v3/users/2f0267a7d5d84439880c6a9ca38dd660 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:17.529183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b382285-c892-4f68-bd3f-e9baa9b18da6 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:17.640612 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3dc124-0696-4cdb-8530-6f3b4af295eb None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:17.667657 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b382285-c892-4f68-bd3f-e9baa9b18da6 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:17.684787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8ec99-ba7c-43b0-b223-78c9749e1e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:17.685929 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8ec99-ba7c-43b0-b223-78c9749e1e3f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:17.686420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8ec99-ba7c-43b0-b223-78c9749e1e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:17.686938 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8ec99-ba7c-43b0-b223-78c9749e1e3f None admin] PATH_INFO: `/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:17.688594 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8ec99-ba7c-43b0-b223-78c9749e1e3f None admin] RBAC: Authorizing `identity:get_project(project_id=79baed4da64f4fc2b256aa6fc529fbd2)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:17.690842 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8ec99-ba7c-43b0-b223-78c9749e1e3f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:17.693874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 91/178] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:55:17 2022] GET /identity/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:17.694930 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8ec99-ba7c-43b0-b223-78c9749e1e3f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:17.709954 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea601866-2222-4a54-a19e-59955783ece4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:17.710647 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea601866-2222-4a54-a19e-59955783ece4 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:17.711003 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea601866-2222-4a54-a19e-59955783ece4 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:17.711388 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea601866-2222-4a54-a19e-59955783ece4 None admin] PATH_INFO: `/v3/users/2f0267a7d5d84439880c6a9ca38dd660` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:17.713313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea601866-2222-4a54-a19e-59955783ece4 None admin] RBAC: Authorizing `identity:get_user(user_id=2f0267a7d5d84439880c6a9ca38dd660)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:17.715052 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea601866-2222-4a54-a19e-59955783ece4 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:17.717627 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 88/179] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:55:17 2022] GET /identity/v3/users/2f0267a7d5d84439880c6a9ca38dd660 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:17.718316 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea601866-2222-4a54-a19e-59955783ece4 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:17.730577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:17.742868 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:17.827987 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8ec99-ba7c-43b0-b223-78c9749e1e3f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:17.833090 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea601866-2222-4a54-a19e-59955783ece4 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:17.869044 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d1e7b-78c3-4f6b-a44f-91c42ec25871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:17.869851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d1e7b-78c3-4f6b-a44f-91c42ec25871 None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:17.870214 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d1e7b-78c3-4f6b-a44f-91c42ec25871 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:17.870588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d1e7b-78c3-4f6b-a44f-91c42ec25871 None admin] PATH_INFO: `/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2/users/ab9f668d87844bedb696a3318a64fab9/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:17.871234 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761d134-54be-44bc-ac1f-cc4103bd0189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:17.872207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761d134-54be-44bc-ac1f-cc4103bd0189 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:17.872688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761d134-54be-44bc-ac1f-cc4103bd0189 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:17.872791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d1e7b-78c3-4f6b-a44f-91c42ec25871 None admin] RBAC: Authorizing `identity:create_grant(project_id=79baed4da64f4fc2b256aa6fc529fbd2, user_id=ab9f668d87844bedb696a3318a64fab9, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:17.873371 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761d134-54be-44bc-ac1f-cc4103bd0189 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:17.874543 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d1e7b-78c3-4f6b-a44f-91c42ec25871 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:17.889536 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 89/180] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:55:17 2022] PUT /identity/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2/users/ab9f668d87844bedb696a3318a64fab9/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:55:17.890337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d1e7b-78c3-4f6b-a44f-91c42ec25871 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:17.970349 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761d134-54be-44bc-ac1f-cc4103bd0189 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:17.971760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761d134-54be-44bc-ac1f-cc4103bd0189 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:17.997206 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:18.064889 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:18.093023 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d1e7b-78c3-4f6b-a44f-91c42ec25871 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:18.151892 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5dfad-05ed-4a66-b269-98cf668889fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:18.152561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5dfad-05ed-4a66-b269-98cf668889fa None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:18.152853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5dfad-05ed-4a66-b269-98cf668889fa None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:18.153132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5dfad-05ed-4a66-b269-98cf668889fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:18.176746 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 92/181] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:55:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 22 23:55:18.182894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761d134-54be-44bc-ac1f-cc4103bd0189 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:18.295032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761d134-54be-44bc-ac1f-cc4103bd0189 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:18.296637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5dfad-05ed-4a66-b269-98cf668889fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:18.298603 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5dfad-05ed-4a66-b269-98cf668889fa None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:18.336561 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42976689-5e9b-463c-b768-da6e01fe22a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:18.336561 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42976689-5e9b-463c-b768-da6e01fe22a1 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:18.336561 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42976689-5e9b-463c-b768-da6e01fe22a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:18.336561 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42976689-5e9b-463c-b768-da6e01fe22a1 None admin] PATH_INFO: `/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:18.341319 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42976689-5e9b-463c-b768-da6e01fe22a1 None admin] RBAC: Authorizing `identity:get_project(project_id=79baed4da64f4fc2b256aa6fc529fbd2)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:18.344738 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42976689-5e9b-463c-b768-da6e01fe22a1 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:18.346769 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 93/182] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:55:18 2022] GET /identity/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:55:18.353527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42976689-5e9b-463c-b768-da6e01fe22a1 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:18.360799 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:18.421198 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:18.479566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 90/183] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:55:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 22 23:55:18.485485 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5dfad-05ed-4a66-b269-98cf668889fa None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:18.530136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42976689-5e9b-463c-b768-da6e01fe22a1 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:18.594435 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d1fb1-5aff-4bb2-acd8-b33fa2f1903d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:18.595053 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d1fb1-5aff-4bb2-acd8-b33fa2f1903d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:18.595335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d1fb1-5aff-4bb2-acd8-b33fa2f1903d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:18.595613 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d1fb1-5aff-4bb2-acd8-b33fa2f1903d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:18.596340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d1fb1-5aff-4bb2-acd8-b33fa2f1903d None admin] RBAC: Adding query filter params (role.id=839a55263e2c4cbaa2f945b75cf298d6, scope.project.id=79baed4da64f4fc2b256aa6fc529fbd2, user.id=2f0267a7d5d84439880c6a9ca38dd660) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:18.596677 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d1fb1-5aff-4bb2-acd8-b33fa2f1903d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:18.598127 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d1fb1-5aff-4bb2-acd8-b33fa2f1903d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:18.614638 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 94/184] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:55:18 2022] GET /identity/v3/role_assignments?user.id=2f0267a7d5d84439880c6a9ca38dd660&scope.project.id=79baed4da64f4fc2b256aa6fc529fbd2&role.id=839a55263e2c4cbaa2f945b75cf298d6 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:18.648435 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5dfad-05ed-4a66-b269-98cf668889fa None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:18.700725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65108d-e6c9-4a97-b8f4-1e94fc5f0ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:18.701689 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65108d-e6c9-4a97-b8f4-1e94fc5f0ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:18.702154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65108d-e6c9-4a97-b8f4-1e94fc5f0ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:18.702635 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65108d-e6c9-4a97-b8f4-1e94fc5f0ef0 None admin] PATH_INFO: `/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:18.709427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff65108d-e6c9-4a97-b8f4-1e94fc5f0ef0 None admin] RBAC: Authorizing `identity:get_project(project_id=79baed4da64f4fc2b256aa6fc529fbd2)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:18.711240 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff65108d-e6c9-4a97-b8f4-1e94fc5f0ef0 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:18.713361 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 91/185] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:55:18 2022] GET /identity/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:18.719524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d1fb1-5aff-4bb2-acd8-b33fa2f1903d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:18.800862 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:18.839644 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d1fb1-5aff-4bb2-acd8-b33fa2f1903d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:18.878295 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cac41c6-4626-479e-bdfd-7582beb5956c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:18.878833 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac41c6-4626-479e-bdfd-7582beb5956c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:18.879120 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac41c6-4626-479e-bdfd-7582beb5956c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:18.879390 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cac41c6-4626-479e-bdfd-7582beb5956c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:18.880124 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac41c6-4626-479e-bdfd-7582beb5956c None admin] RBAC: Adding query filter params (role.id=cf8116e4e82748ac872d8763d113ba61, scope.project.id=79baed4da64f4fc2b256aa6fc529fbd2, user.id=2f0267a7d5d84439880c6a9ca38dd660) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:18.880439 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac41c6-4626-479e-bdfd-7582beb5956c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:18.882059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cac41c6-4626-479e-bdfd-7582beb5956c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:18.890186 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 95/186] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:55:18 2022] GET /identity/v3/role_assignments?user.id=2f0267a7d5d84439880c6a9ca38dd660&scope.project.id=79baed4da64f4fc2b256aa6fc529fbd2&role.id=cf8116e4e82748ac872d8763d113ba61 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:19.644801 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad65708-f9dd-49c5-b334-3ce411b94ade None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:19.645315 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad65708-f9dd-49c5-b334-3ce411b94ade None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:19.645390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad65708-f9dd-49c5-b334-3ce411b94ade None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:19.646824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 92/187] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:19.651797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5faa97-5b61-47bc-8433-314f847298e2 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:19.652190 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5faa97-5b61-47bc-8433-314f847298e2 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:19.652355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5faa97-5b61-47bc-8433-314f847298e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:19.673903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-cc5faa97-5b61-47bc-8433-314f847298e2 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:19.675169 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-cc5faa97-5b61-47bc-8433-314f847298e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:19.747879 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 96/188] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:19.796769 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181e515-3e09-4438-a069-52718378ab60 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:19.797061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181e515-3e09-4438-a069-52718378ab60 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:19.797922 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181e515-3e09-4438-a069-52718378ab60 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:19.799420 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 93/189] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:19.804534 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8a929-e486-4a23-a4f5-f5a25262824c None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:19.804839 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8a929-e486-4a23-a4f5-f5a25262824c None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:19.805124 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8a929-e486-4a23-a4f5-f5a25262824c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:19.823603 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-5ed8a929-e486-4a23-a4f5-f5a25262824c None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:19.824823 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5ed8a929-e486-4a23-a4f5-f5a25262824c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:19.880583 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:19.893035 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 97/190] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:20.023052 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181e515-3e09-4438-a069-52718378ab60 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:20.060035 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:20.138400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181e515-3e09-4438-a069-52718378ab60 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:20.166925 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8a929-e486-4a23-a4f5-f5a25262824c None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:20.181042 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c2758-1255-4c00-a4d7-d242f37783e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:20.181683 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c2758-1255-4c00-a4d7-d242f37783e5 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:20.182058 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c2758-1255-4c00-a4d7-d242f37783e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:20.182334 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c2758-1255-4c00-a4d7-d242f37783e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:20.183499 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c2758-1255-4c00-a4d7-d242f37783e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:20.184959 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c2758-1255-4c00-a4d7-d242f37783e5 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:20.186526 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 94/191] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:20.192518 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c2758-1255-4c00-a4d7-d242f37783e5 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:20.273987 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8a929-e486-4a23-a4f5-f5a25262824c None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:20.299694 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c2758-1255-4c00-a4d7-d242f37783e5 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:20.301420 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:20.308078 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2bc9ff-5748-4022-87f8-38917ee6664e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:20.308739 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2bc9ff-5748-4022-87f8-38917ee6664e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:20.309027 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2bc9ff-5748-4022-87f8-38917ee6664e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:20.309311 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2bc9ff-5748-4022-87f8-38917ee6664e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:20.321453 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2bc9ff-5748-4022-87f8-38917ee6664e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:20.326359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-2b2bc9ff-5748-4022-87f8-38917ee6664e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 22 23:55:20.326359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:20.326359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:20.326359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:20.326359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:20.326359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:20.326359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:20.326359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:20.326359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:20.326359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:20.326359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:20.326359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:20.326359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:20.326359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:20.326359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:20.326359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:20.327391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:20.327391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:20.327391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:20.327391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:20.327391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:20.327391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:20.327391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:20.327391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:20.327391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:20.327391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:20.327391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:20.327391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:20.327391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:20.327391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:20.327391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:20.327391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:20.328197 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:20.328197 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:20.328197 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:20.328197 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:20.328197 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:20.328197 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:20.328197 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:20.328197 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:20.328197 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:20.328197 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:20.328197 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 22 23:55:20.328197 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:20.328197 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 98/192] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:20.334461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7f51f-2e08-4484-a4d4-90c30e625f73 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:20.335267 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7f51f-2e08-4484-a4d4-90c30e625f73 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:20.335418 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921f672-f7c9-4e61-bd69-d5438fb860d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:20.335554 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7f51f-2e08-4484-a4d4-90c30e625f73 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:20.336026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921f672-f7c9-4e61-bd69-d5438fb860d9 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:20.336309 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921f672-f7c9-4e61-bd69-d5438fb860d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:20.336624 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921f672-f7c9-4e61-bd69-d5438fb860d9 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:20.336980 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 99/193] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:20.337349 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921f672-f7c9-4e61-bd69-d5438fb860d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:20.337554 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7f51f-2e08-4484-a4d4-90c30e625f73 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:20.338890 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921f672-f7c9-4e61-bd69-d5438fb860d9 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:20.360396 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 95/194] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Oct 22 23:55:20 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 22 23:55:20.361642 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd243d-93cd-49d7-b18d-f43722f3b8a4 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:20.361930 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd243d-93cd-49d7-b18d-f43722f3b8a4 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:20.362208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd243d-93cd-49d7-b18d-f43722f3b8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:20.389532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-82fd243d-93cd-49d7-b18d-f43722f3b8a4 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:20.390903 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-82fd243d-93cd-49d7-b18d-f43722f3b8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:20.402782 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:20.464972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7f51f-2e08-4484-a4d4-90c30e625f73 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:20.480525 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 96/195] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:20.510991 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31674ac2-a7d3-4f17-aaef-d69a8fd13a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:20.511714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31674ac2-a7d3-4f17-aaef-d69a8fd13a74 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:20.512054 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31674ac2-a7d3-4f17-aaef-d69a8fd13a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:20.512400 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31674ac2-a7d3-4f17-aaef-d69a8fd13a74 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:20.513175 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31674ac2-a7d3-4f17-aaef-d69a8fd13a74 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:20.513572 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31674ac2-a7d3-4f17-aaef-d69a8fd13a74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:20.515039 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31674ac2-a7d3-4f17-aaef-d69a8fd13a74 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:20.524789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 100/196] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Oct 22 23:55:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:20.530126 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fd243d-93cd-49d7-b18d-f43722f3b8a4 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:20.623371 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:20.638440 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fd243d-93cd-49d7-b18d-f43722f3b8a4 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:20.679133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758a2700-f6e0-4548-bdf9-1551249724b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:20.680606 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a2700-f6e0-4548-bdf9-1551249724b0 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:20.681121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a2700-f6e0-4548-bdf9-1551249724b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:20.681616 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a2700-f6e0-4548-bdf9-1551249724b0 None admin] PATH_INFO: `/v3/users/ebc861367d5a4782965564280ef293c7` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:20.696382 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758a2700-f6e0-4548-bdf9-1551249724b0 None admin] RBAC: Authorizing `identity:get_user(user_id=ebc861367d5a4782965564280ef293c7)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:20.698131 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758a2700-f6e0-4548-bdf9-1551249724b0 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:20.700010 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 97/197] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:55:20 2022] GET /identity/v3/users/ebc861367d5a4782965564280ef293c7 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:20.706388 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31674ac2-a7d3-4f17-aaef-d69a8fd13a74 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:20.748853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758a2700-f6e0-4548-bdf9-1551249724b0 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:20.828232 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31674ac2-a7d3-4f17-aaef-d69a8fd13a74 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:20.842995 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:20.872224 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758a2700-f6e0-4548-bdf9-1551249724b0 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:20.873986 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5783688-698b-4e77-86de-896d1e5fc4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:20.874647 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5783688-698b-4e77-86de-896d1e5fc4ff None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:20.875859 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5783688-698b-4e77-86de-896d1e5fc4ff None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:20.876140 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5783688-698b-4e77-86de-896d1e5fc4ff None admin] PATH_INFO: `/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:20.877340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5783688-698b-4e77-86de-896d1e5fc4ff None admin] RBAC: Authorizing `identity:get_project(project_id=79baed4da64f4fc2b256aa6fc529fbd2)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:20.878789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5783688-698b-4e77-86de-896d1e5fc4ff None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:20.880456 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 101/198] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:55:20 2022] GET /identity/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:20.886102 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5783688-698b-4e77-86de-896d1e5fc4ff None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:20.914057 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba452f0-126a-4d1e-af75-84ebc5616cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:20.914751 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba452f0-126a-4d1e-af75-84ebc5616cab None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:20.915036 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba452f0-126a-4d1e-af75-84ebc5616cab None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:20.915310 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba452f0-126a-4d1e-af75-84ebc5616cab None admin] PATH_INFO: `/v3/roles/member` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:20.926840 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba452f0-126a-4d1e-af75-84ebc5616cab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:20.931883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-1ba452f0-126a-4d1e-af75-84ebc5616cab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:20.931883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:20.931883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:20.931883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:20.931883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:20.931883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:20.931883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:20.931883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:20.931883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:20.931883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:20.931883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:20.931883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:20.931883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:20.931883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:20.931883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:20.931883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:20.932825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:20.932825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:20.932825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:20.932825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:20.932825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:20.932825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:20.932825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:20.932825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:20.932825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:20.932825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:20.932825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:20.932825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:20.932825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:20.932825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:20.932825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:20.932825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:20.934007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:20.934007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:20.934007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:20.934007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:20.934007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:20.934007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:20.934007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:20.934007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:20.934007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:20.934007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:20.934007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:20.934007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:20.934007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 98/199] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:55:20.939059 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba452f0-126a-4d1e-af75-84ebc5616cab None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:20.951292 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:21.007986 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5783688-698b-4e77-86de-896d1e5fc4ff None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:21.046973 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd3a1b-2b3a-4cba-ae18-dc3ae20bb539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:21.048272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd3a1b-2b3a-4cba-ae18-dc3ae20bb539 None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:21.048587 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd3a1b-2b3a-4cba-ae18-dc3ae20bb539 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:21.048869 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd3a1b-2b3a-4cba-ae18-dc3ae20bb539 None admin] PATH_INFO: `/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2/users/ebc861367d5a4782965564280ef293c7/roles/159c6380cd984e998bb6a97ed7d98dac` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:21.051164 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd3a1b-2b3a-4cba-ae18-dc3ae20bb539 None admin] RBAC: Authorizing `identity:create_grant(project_id=79baed4da64f4fc2b256aa6fc529fbd2, user_id=ebc861367d5a4782965564280ef293c7, role_id=159c6380cd984e998bb6a97ed7d98dac)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:21.052560 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd3a1b-2b3a-4cba-ae18-dc3ae20bb539 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:21.068509 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 102/200] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:55:20 2022] PUT /identity/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2/users/ebc861367d5a4782965564280ef293c7/roles/159c6380cd984e998bb6a97ed7d98dac => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 22 23:55:21.077021 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba452f0-126a-4d1e-af75-84ebc5616cab None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:21.117054 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10152706-2758-40c0-8caf-c496760f2d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:21.117768 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10152706-2758-40c0-8caf-c496760f2d79 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:21.118130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10152706-2758-40c0-8caf-c496760f2d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:21.118481 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10152706-2758-40c0-8caf-c496760f2d79 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:21.119284 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10152706-2758-40c0-8caf-c496760f2d79 None admin] RBAC: Adding query filter params (name=member) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:21.119658 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10152706-2758-40c0-8caf-c496760f2d79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:21.121365 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10152706-2758-40c0-8caf-c496760f2d79 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:21.131688 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 99/201] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Oct 22 23:55:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:21.137871 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd3a1b-2b3a-4cba-ae18-dc3ae20bb539 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:21.185996 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:21.254780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd3a1b-2b3a-4cba-ae18-dc3ae20bb539 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:21.291976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6cbdc-74fe-4022-8aac-acf9a2bd5516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:21.292660 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6cbdc-74fe-4022-8aac-acf9a2bd5516 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:21.292946 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6cbdc-74fe-4022-8aac-acf9a2bd5516 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:21.293221 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6cbdc-74fe-4022-8aac-acf9a2bd5516 None admin] PATH_INFO: `/v3/users/ab9f668d87844bedb696a3318a64fab9` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:21.294358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6cbdc-74fe-4022-8aac-acf9a2bd5516 None admin] RBAC: Authorizing `identity:get_user(user_id=ab9f668d87844bedb696a3318a64fab9)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:21.295789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6cbdc-74fe-4022-8aac-acf9a2bd5516 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:21.297471 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 103/202] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:55:21 2022] GET /identity/v3/users/ab9f668d87844bedb696a3318a64fab9 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:21.303235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10152706-2758-40c0-8caf-c496760f2d79 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:21.373534 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:21.410616 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10152706-2758-40c0-8caf-c496760f2d79 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:21.450747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa975a10-6eb4-46fe-83bf-33419077cd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:21.451423 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa975a10-6eb4-46fe-83bf-33419077cd24 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:21.451712 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa975a10-6eb4-46fe-83bf-33419077cd24 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:21.452093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa975a10-6eb4-46fe-83bf-33419077cd24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:21.534183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa975a10-6eb4-46fe-83bf-33419077cd24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:21.536011 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa975a10-6eb4-46fe-83bf-33419077cd24 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:21.602284 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:21.638608 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 100/203] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:55:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 22 23:55:21.644451 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6cbdc-74fe-4022-8aac-acf9a2bd5516 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:21.748406 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6cbdc-74fe-4022-8aac-acf9a2bd5516 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:21.758845 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:21.784736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5ba11-2027-4a47-a882-7f1e39e28cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:21.785377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5ba11-2027-4a47-a882-7f1e39e28cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:21.785660 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5ba11-2027-4a47-a882-7f1e39e28cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:21.785933 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5ba11-2027-4a47-a882-7f1e39e28cc0 None admin] PATH_INFO: `/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:21.787068 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5ba11-2027-4a47-a882-7f1e39e28cc0 None admin] RBAC: Authorizing `identity:get_project(project_id=79baed4da64f4fc2b256aa6fc529fbd2)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:21.788531 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5ba11-2027-4a47-a882-7f1e39e28cc0 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:21.790513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 104/204] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:55:21 2022] GET /identity/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:21.796320 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa975a10-6eb4-46fe-83bf-33419077cd24 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:21.901176 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa975a10-6eb4-46fe-83bf-33419077cd24 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:21.937526 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf333c8-67e0-4c86-9dc6-be44d926f04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:21.939112 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf333c8-67e0-4c86-9dc6-be44d926f04a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:21.939610 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf333c8-67e0-4c86-9dc6-be44d926f04a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:21.940149 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf333c8-67e0-4c86-9dc6-be44d926f04a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:21.941424 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf333c8-67e0-4c86-9dc6-be44d926f04a None admin] RBAC: Adding query filter params (role.id=4bd2874c95634a50aac9fccbddc7c674, scope.project.id=79baed4da64f4fc2b256aa6fc529fbd2, user.id=ab9f668d87844bedb696a3318a64fab9) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:21.941939 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf333c8-67e0-4c86-9dc6-be44d926f04a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:21.943603 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf333c8-67e0-4c86-9dc6-be44d926f04a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:21.950219 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 101/205] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:55:21 2022] GET /identity/v3/role_assignments?user.id=ab9f668d87844bedb696a3318a64fab9&scope.project.id=79baed4da64f4fc2b256aa6fc529fbd2&role.id=4bd2874c95634a50aac9fccbddc7c674 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:22.705314 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cc8b9-8342-4e7a-9382-a0883a575636 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:22.705695 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cc8b9-8342-4e7a-9382-a0883a575636 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:22.705872 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cc8b9-8342-4e7a-9382-a0883a575636 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:22.707346 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 105/206] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 22 23:55:22.712269 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933fd179-2333-4112-ba2a-10637324276b None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:22.712585 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933fd179-2333-4112-ba2a-10637324276b None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:22.712865 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933fd179-2333-4112-ba2a-10637324276b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:22.731576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-933fd179-2333-4112-ba2a-10637324276b None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:22.732778 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-933fd179-2333-4112-ba2a-10637324276b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:22.759308 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:22.774586 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960a0f0-2f69-46a9-a1d7-f8f7c02a9ce0 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:22.775092 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960a0f0-2f69-46a9-a1d7-f8f7c02a9ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:22.775181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960a0f0-2f69-46a9-a1d7-f8f7c02a9ce0 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:22.776615 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 106/207] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 22 23:55:22.781474 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd3dad-86da-4dcf-81bf-c0690331f241 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:22.782499 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd3dad-86da-4dcf-81bf-c0690331f241 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:22.782736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd3dad-86da-4dcf-81bf-c0690331f241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:22.801353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-89fd3dad-86da-4dcf-81bf-c0690331f241 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:22.802541 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-89fd3dad-86da-4dcf-81bf-c0690331f241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:22.804543 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 102/208] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:22.870197 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 107/209] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 22 23:55:23.071760 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933fd179-2333-4112-ba2a-10637324276b None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:23.139250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd3dad-86da-4dcf-81bf-c0690331f241 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:23.177174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933fd179-2333-4112-ba2a-10637324276b None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:23.212458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12750652-0ed0-4fc8-9987-3faa1a2b83cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:23.213355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12750652-0ed0-4fc8-9987-3faa1a2b83cc None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:23.213792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12750652-0ed0-4fc8-9987-3faa1a2b83cc None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:23.214233 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12750652-0ed0-4fc8-9987-3faa1a2b83cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:23.224973 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12750652-0ed0-4fc8-9987-3faa1a2b83cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:23.229809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-12750652-0ed0-4fc8-9987-3faa1a2b83cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:23.229809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:23.229809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:23.229809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:23.229809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:23.229809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:23.229809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:23.229809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:23.229809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:23.229809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:23.229809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:23.229809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:23.229809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:23.229809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:23.229809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:23.229809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:23.230705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:23.230705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:23.230705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:23.230705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:23.230705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:23.230705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:23.230705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:23.230705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:23.230705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:23.230705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:23.230705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:23.230705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:23.230705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:23.230705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:23.230705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:23.230705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:23.231596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:23.231596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:23.231596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:23.231596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:23.231596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:23.231596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:23.231596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:23.231596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:23.231596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:23.231596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:23.231596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:23.231596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:23.231596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 103/210] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:23.238275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12750652-0ed0-4fc8-9987-3faa1a2b83cc None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:23.245739 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd3dad-86da-4dcf-81bf-c0690331f241 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:23.255636 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:23.266033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:23.288012 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc8f5b-e73b-432e-81c4-4d1043efd4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:23.288415 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc8f5b-e73b-432e-81c4-4d1043efd4c4 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:23.288771 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc8f5b-e73b-432e-81c4-4d1043efd4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:23.289073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc8f5b-e73b-432e-81c4-4d1043efd4c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:23.290370 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc8f5b-e73b-432e-81c4-4d1043efd4c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:23.291853 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc8f5b-e73b-432e-81c4-4d1043efd4c4 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:23.294159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 108/211] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:55:23.300348 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc8f5b-e73b-432e-81c4-4d1043efd4c4 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:23.347742 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12750652-0ed0-4fc8-9987-3faa1a2b83cc None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:23.382697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a678b-6cc5-495c-b343-152922c82e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:23.383412 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a678b-6cc5-495c-b343-152922c82e9b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:23.383690 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a678b-6cc5-495c-b343-152922c82e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:23.383966 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a678b-6cc5-495c-b343-152922c82e9b None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:23.384662 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a678b-6cc5-495c-b343-152922c82e9b None admin] RBAC: Adding query filter params (name=member) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:23.384963 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a678b-6cc5-495c-b343-152922c82e9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:23.386324 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a678b-6cc5-495c-b343-152922c82e9b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:23.396525 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 104/212] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Oct 22 23:55:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:55:23.396525 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe2981-bb69-410c-a6a5-382336cd67bb None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:23.396525 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe2981-bb69-410c-a6a5-382336cd67bb None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:23.396525 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe2981-bb69-410c-a6a5-382336cd67bb None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:23.397437 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 105/213] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:23.400091 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe2981-bb69-410c-a6a5-382336cd67bb None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:23.408634 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc8f5b-e73b-432e-81c4-4d1043efd4c4 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:23.445905 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fadae1-af5a-4ce9-a4fc-c3303c1b818c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:23.446516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fadae1-af5a-4ce9-a4fc-c3303c1b818c None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:23.446802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fadae1-af5a-4ce9-a4fc-c3303c1b818c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:23.447076 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fadae1-af5a-4ce9-a4fc-c3303c1b818c None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:23.447730 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fadae1-af5a-4ce9-a4fc-c3303c1b818c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:23.449197 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fadae1-af5a-4ce9-a4fc-c3303c1b818c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:23.480867 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 109/214] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Oct 22 23:55:23 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 22 23:55:23.482845 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375d7a3-f998-42d0-846f-427d8d6e0bcc None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:23.483139 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375d7a3-f998-42d0-846f-427d8d6e0bcc None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:23.483415 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375d7a3-f998-42d0-846f-427d8d6e0bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:23.493203 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:23.512132 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-7375d7a3-f998-42d0-846f-427d8d6e0bcc None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:23.513180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe2981-bb69-410c-a6a5-382336cd67bb None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:23.513411 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-7375d7a3-f998-42d0-846f-427d8d6e0bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:23.543396 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:23.562720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667491ac-50c9-49cc-8c03-f80356d574a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:23.563498 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667491ac-50c9-49cc-8c03-f80356d574a6 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:23.563860 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667491ac-50c9-49cc-8c03-f80356d574a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:23.564188 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667491ac-50c9-49cc-8c03-f80356d574a6 None admin] PATH_INFO: `/v3/users/459e702ae60f4bc4a0ea0aa668e14160` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:23.580225 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667491ac-50c9-49cc-8c03-f80356d574a6 None admin] RBAC: Authorizing `identity:get_user(user_id=459e702ae60f4bc4a0ea0aa668e14160)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:23.581732 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667491ac-50c9-49cc-8c03-f80356d574a6 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:23.583460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 106/215] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:55:23 2022] GET /identity/v3/users/459e702ae60f4bc4a0ea0aa668e14160 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:55:23.589498 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667491ac-50c9-49cc-8c03-f80356d574a6 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:23.599194 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 110/216] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:23.716115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667491ac-50c9-49cc-8c03-f80356d574a6 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:23.754992 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9fbe42-03aa-4b19-a008-57c8a2ba107f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:23.755608 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fbe42-03aa-4b19-a008-57c8a2ba107f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:23.755890 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fbe42-03aa-4b19-a008-57c8a2ba107f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:23.756183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fbe42-03aa-4b19-a008-57c8a2ba107f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:23.756949 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9fbe42-03aa-4b19-a008-57c8a2ba107f None admin] RBAC: Adding query filter params (role.id=4bd2874c95634a50aac9fccbddc7c674, user.id=459e702ae60f4bc4a0ea0aa668e14160, scope.system=all) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:23.757256 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9fbe42-03aa-4b19-a008-57c8a2ba107f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:23.758714 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9fbe42-03aa-4b19-a008-57c8a2ba107f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:23.764157 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 107/217] 10.222.0.13 () {56 vars in 1308 bytes} [Sat Oct 22 23:55:23 2022] GET /identity/v3/role_assignments?user.id=459e702ae60f4bc4a0ea0aa668e14160&scope.system=all&role.id=4bd2874c95634a50aac9fccbddc7c674 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:55:23.897241 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375d7a3-f998-42d0-846f-427d8d6e0bcc None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:24.017532 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375d7a3-f998-42d0-846f-427d8d6e0bcc None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:24.056518 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29789995-2855-49b1-80b9-e8e8795ff63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:24.057490 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29789995-2855-49b1-80b9-e8e8795ff63e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:24.057950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29789995-2855-49b1-80b9-e8e8795ff63e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:24.058401 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29789995-2855-49b1-80b9-e8e8795ff63e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:24.070058 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29789995-2855-49b1-80b9-e8e8795ff63e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:24.075276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-29789995-2855-49b1-80b9-e8e8795ff63e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 22 23:55:24.075276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:24.075276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:24.075276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:24.075276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:24.075276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:24.075276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:24.075276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:24.075276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:24.075276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:24.075276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:24.075276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:24.075276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:24.075276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:24.075276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:24.075276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:24.076185 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:24.076185 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:24.076185 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:24.076185 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:24.076185 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:24.076185 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:24.076185 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:24.076185 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:24.076185 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:24.076185 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:24.076185 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:24.076185 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:24.076185 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:24.076185 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:24.076185 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:24.076185 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:24.077003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:24.077003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:24.077003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:24.077003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:24.077003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:24.077003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:24.077003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:24.077003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:24.077003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:24.077003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:24.077003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 22 23:55:24.077003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:24.079049 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 111/218] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:24.085274 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9fbe42-03aa-4b19-a008-57c8a2ba107f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:24.096139 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:24.207961 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9fbe42-03aa-4b19-a008-57c8a2ba107f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:24.246284 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e99de9-b137-4edd-bb56-6175cae4f365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:24.247137 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e99de9-b137-4edd-bb56-6175cae4f365 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:24.247561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e99de9-b137-4edd-bb56-6175cae4f365 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:24.247976 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e99de9-b137-4edd-bb56-6175cae4f365 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:24.248813 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e99de9-b137-4edd-bb56-6175cae4f365 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:24.249279 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e99de9-b137-4edd-bb56-6175cae4f365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:24.251441 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e99de9-b137-4edd-bb56-6175cae4f365 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:24.265326 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 108/219] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Oct 22 23:55:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:55:24.273790 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29789995-2855-49b1-80b9-e8e8795ff63e None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:24.303122 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:24.396626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29789995-2855-49b1-80b9-e8e8795ff63e None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:24.437548 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f383bb0-929b-4800-9f47-656c94a5d29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:24.439059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f383bb0-929b-4800-9f47-656c94a5d29d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:24.439772 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f383bb0-929b-4800-9f47-656c94a5d29d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:24.440470 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f383bb0-929b-4800-9f47-656c94a5d29d None admin] PATH_INFO: `/v3/users/ebc861367d5a4782965564280ef293c7` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:24.442536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f383bb0-929b-4800-9f47-656c94a5d29d None admin] RBAC: Authorizing `identity:get_user(user_id=ebc861367d5a4782965564280ef293c7)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:24.444248 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f383bb0-929b-4800-9f47-656c94a5d29d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:24.446395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 112/220] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:55:24 2022] GET /identity/v3/users/ebc861367d5a4782965564280ef293c7 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:24.452177 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e99de9-b137-4edd-bb56-6175cae4f365 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:24.527903 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:24.570347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e99de9-b137-4edd-bb56-6175cae4f365 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:24.611516 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71208005-4263-4423-8fb7-efa7e1ab60b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:24.612137 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71208005-4263-4423-8fb7-efa7e1ab60b5 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:24.612425 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71208005-4263-4423-8fb7-efa7e1ab60b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:24.612731 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71208005-4263-4423-8fb7-efa7e1ab60b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:24.690332 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71208005-4263-4423-8fb7-efa7e1ab60b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:24.692052 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71208005-4263-4423-8fb7-efa7e1ab60b5 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:24.769959 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:24.807852 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 109/221] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:55:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 22 23:55:24.814136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f383bb0-929b-4800-9f47-656c94a5d29d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:24.905043 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:24.919050 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f383bb0-929b-4800-9f47-656c94a5d29d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:24.955437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ee921-1cb4-4c4c-9454-7c5b90eaea5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:24.956072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ee921-1cb4-4c4c-9454-7c5b90eaea5e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:24.956351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ee921-1cb4-4c4c-9454-7c5b90eaea5e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:24.956655 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ee921-1cb4-4c4c-9454-7c5b90eaea5e None admin] PATH_INFO: `/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:24.957774 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ee921-1cb4-4c4c-9454-7c5b90eaea5e None admin] RBAC: Authorizing `identity:get_project(project_id=79baed4da64f4fc2b256aa6fc529fbd2)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:24.959285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ee921-1cb4-4c4c-9454-7c5b90eaea5e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:24.961246 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 113/222] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:55:24 2022] GET /identity/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:24.967129 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71208005-4263-4423-8fb7-efa7e1ab60b5 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:25.075926 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71208005-4263-4423-8fb7-efa7e1ab60b5 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:25.115565 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9a974-9a62-479f-8f46-575a981d7b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:25.116200 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9a974-9a62-479f-8f46-575a981d7b23 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:25.116460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9a974-9a62-479f-8f46-575a981d7b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:25.116766 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9a974-9a62-479f-8f46-575a981d7b23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:25.117503 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca9a974-9a62-479f-8f46-575a981d7b23 None admin] RBAC: Adding query filter params (role.id=159c6380cd984e998bb6a97ed7d98dac, scope.project.id=79baed4da64f4fc2b256aa6fc529fbd2, user.id=ebc861367d5a4782965564280ef293c7) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:25.117817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca9a974-9a62-479f-8f46-575a981d7b23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:25.119248 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca9a974-9a62-479f-8f46-575a981d7b23 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:25.125951 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 110/223] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:55:24 2022] GET /identity/v3/role_assignments?user.id=ebc861367d5a4782965564280ef293c7&scope.project.id=79baed4da64f4fc2b256aa6fc529fbd2&role.id=159c6380cd984e998bb6a97ed7d98dac => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:25.872261 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83104419-9aad-4747-9910-058f3b17101a None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:25.872695 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83104419-9aad-4747-9910-058f3b17101a None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:25.872884 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83104419-9aad-4747-9910-058f3b17101a None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:25.874427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 114/224] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:25.880641 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa718707-2b1b-420c-99b3-804725f60abf None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:25.881019 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa718707-2b1b-420c-99b3-804725f60abf None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:25.881372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa718707-2b1b-420c-99b3-804725f60abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:25.899898 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-fa718707-2b1b-420c-99b3-804725f60abf None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:25.901122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-fa718707-2b1b-420c-99b3-804725f60abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:25.927641 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:25.973351 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 111/225] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 22 23:55:26.061745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c49ff-a5fe-4dce-a018-4a2f33a2a4a3 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:26.062365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c49ff-a5fe-4dce-a018-4a2f33a2a4a3 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:26.062365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c49ff-a5fe-4dce-a018-4a2f33a2a4a3 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:26.064582 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 115/226] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:26.069961 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b9ca7-4332-484e-a470-1a9f2fd7616a None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:26.070253 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b9ca7-4332-484e-a470-1a9f2fd7616a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:26.070530 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b9ca7-4332-484e-a470-1a9f2fd7616a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:26.089410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-518b9ca7-4332-484e-a470-1a9f2fd7616a None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:26.090589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-518b9ca7-4332-484e-a470-1a9f2fd7616a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:26.104690 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce55e93-6a6e-471e-906f-07a2538a5724 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:26.105817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce55e93-6a6e-471e-906f-07a2538a5724 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:26.106453 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce55e93-6a6e-471e-906f-07a2538a5724 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:26.109018 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 116/227] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:26.115123 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e87eb8-9868-4526-befd-0f5ed8ddf4ee None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:26.115546 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e87eb8-9868-4526-befd-0f5ed8ddf4ee None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:26.116011 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e87eb8-9868-4526-befd-0f5ed8ddf4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:26.135446 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-56e87eb8-9868-4526-befd-0f5ed8ddf4ee None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:26.136599 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-56e87eb8-9868-4526-befd-0f5ed8ddf4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:26.167331 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 112/228] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:26.209902 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 117/229] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 22 23:55:26.250251 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b9ca7-4332-484e-a470-1a9f2fd7616a None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:26.346269 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:26.355721 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b9ca7-4332-484e-a470-1a9f2fd7616a None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:26.392443 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4a631-1ae8-47c6-b686-3c4f19bb5869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:26.393036 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4a631-1ae8-47c6-b686-3c4f19bb5869 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:26.393318 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4a631-1ae8-47c6-b686-3c4f19bb5869 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:26.393592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd4a631-1ae8-47c6-b686-3c4f19bb5869 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:26.403401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd4a631-1ae8-47c6-b686-3c4f19bb5869 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:26.408726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-6fd4a631-1ae8-47c6-b686-3c4f19bb5869 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 22 23:55:26.408726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:26.408726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:26.408726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:26.408726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:26.408726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:26.408726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:26.408726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:26.408726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:26.408726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:26.408726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:26.408726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:26.408726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:26.408726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:26.408726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:26.408726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:26.409609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:26.409609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:26.409609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:26.409609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:26.409609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:26.409609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:26.409609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:26.409609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:26.409609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:26.409609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:26.409609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:26.409609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:26.409609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:26.409609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:26.409609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:26.409609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:26.410405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:26.410405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:26.410405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:26.410405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:26.410405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:26.410405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:26.410405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:26.410405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:26.410405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:26.410405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:26.410405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 22 23:55:26.410405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:26.410405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 113/230] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:26.415693 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e87eb8-9868-4526-befd-0f5ed8ddf4ee None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:26.443565 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4a631-1ae8-47c6-b686-3c4f19bb5869 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:26.508805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:26.524264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e87eb8-9868-4526-befd-0f5ed8ddf4ee None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:26.545474 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd4a631-1ae8-47c6-b686-3c4f19bb5869 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:26.558212 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50840a3-bacb-4620-b168-c21f9a4d60b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:26.558817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50840a3-bacb-4620-b168-c21f9a4d60b8 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:26.559099 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50840a3-bacb-4620-b168-c21f9a4d60b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:26.559371 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50840a3-bacb-4620-b168-c21f9a4d60b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:26.560022 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50840a3-bacb-4620-b168-c21f9a4d60b8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:26.560324 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50840a3-bacb-4620-b168-c21f9a4d60b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:26.561732 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50840a3-bacb-4620-b168-c21f9a4d60b8 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:26.570629 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 118/231] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Oct 22 23:55:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:26.570629 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50840a3-bacb-4620-b168-c21f9a4d60b8 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:26.583534 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7c463-b310-499e-822f-ebfc2489b8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:26.584172 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7c463-b310-499e-822f-ebfc2489b8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:26.584459 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7c463-b310-499e-822f-ebfc2489b8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:26.584774 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7c463-b310-499e-822f-ebfc2489b8f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:26.585884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe7c463-b310-499e-822f-ebfc2489b8f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:26.587339 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe7c463-b310-499e-822f-ebfc2489b8f5 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:26.588953 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 114/232] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:55:26.589717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7c463-b310-499e-822f-ebfc2489b8f5 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:26.611434 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:26.676663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50840a3-bacb-4620-b168-c21f9a4d60b8 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:26.690999 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7c463-b310-499e-822f-ebfc2489b8f5 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:26.710748 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2298b8-2ba3-4f1b-a3d1-d741fa2bee18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:26.711382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2298b8-2ba3-4f1b-a3d1-d741fa2bee18 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:26.711666 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2298b8-2ba3-4f1b-a3d1-d741fa2bee18 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:26.711961 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2298b8-2ba3-4f1b-a3d1-d741fa2bee18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:26.722104 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2298b8-2ba3-4f1b-a3d1-d741fa2bee18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:26.723597 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2bb74-1ab0-4703-8ed9-0aa23bc9e676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:26.724230 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2bb74-1ab0-4703-8ed9-0aa23bc9e676 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:26.724544 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2bb74-1ab0-4703-8ed9-0aa23bc9e676 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:26.724829 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2bb74-1ab0-4703-8ed9-0aa23bc9e676 None admin] PATH_INFO: `/v3/users/5616f86f495348e6b7022d7ddb16df1c` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:26.727427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-3f2298b8-2ba3-4f1b-a3d1-d741fa2bee18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:26.727427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:26.727427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:26.727427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:26.727427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:26.727427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:26.727427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:26.727427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:26.727427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:26.727427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:26.727427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:26.727427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:26.727427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:26.727427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:26.727427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:26.727427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:26.728469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:26.728469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:26.728469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:26.728469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:26.728469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:26.728469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:26.728469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:26.728469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:26.728469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:26.728469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:26.728469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:26.728469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:26.728469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:26.728469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:26.728469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:26.728469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:26.729299 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:26.729299 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:26.729299 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:26.729299 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:26.729299 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:26.729299 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:26.729299 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:26.729299 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:26.729299 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:26.729299 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:26.729299 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:26.729299 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:26.729299 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 119/233] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:26.729299 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2298b8-2ba3-4f1b-a3d1-d741fa2bee18 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:26.739919 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:26.741560 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2bb74-1ab0-4703-8ed9-0aa23bc9e676 None admin] RBAC: Authorizing `identity:get_user(user_id=5616f86f495348e6b7022d7ddb16df1c)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:26.743000 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2bb74-1ab0-4703-8ed9-0aa23bc9e676 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:26.744696 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 115/234] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:55:26 2022] GET /identity/v3/users/5616f86f495348e6b7022d7ddb16df1c => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:26.745466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2bb74-1ab0-4703-8ed9-0aa23bc9e676 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:26.821577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:26.828806 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2298b8-2ba3-4f1b-a3d1-d741fa2bee18 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:26.843178 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2bb74-1ab0-4703-8ed9-0aa23bc9e676 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:26.861619 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3253b6-1d82-47fe-805c-6091213b2811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:26.862228 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3253b6-1d82-47fe-805c-6091213b2811 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:26.862509 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3253b6-1d82-47fe-805c-6091213b2811 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:26.862783 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3253b6-1d82-47fe-805c-6091213b2811 None admin] PATH_INFO: `/v3/groups` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:26.863410 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3253b6-1d82-47fe-805c-6091213b2811 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:26.864870 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3253b6-1d82-47fe-805c-6091213b2811 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:26.877439 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb5a8b-435f-45c7-b1a5-16e9864afbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:26.878037 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb5a8b-435f-45c7-b1a5-16e9864afbc0 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:26.878318 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb5a8b-435f-45c7-b1a5-16e9864afbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:26.878590 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb5a8b-435f-45c7-b1a5-16e9864afbc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:26.879246 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb5a8b-435f-45c7-b1a5-16e9864afbc0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:26.879547 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb5a8b-435f-45c7-b1a5-16e9864afbc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:26.881284 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb5a8b-435f-45c7-b1a5-16e9864afbc0 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:26.881636 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 120/235] 10.222.0.13 () {60 vars in 1134 bytes} [Sat Oct 22 23:55:26 2022] POST /identity/v3/groups => generated 229 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:55:26.882736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3253b6-1d82-47fe-805c-6091213b2811 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:26.889960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 116/236] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Oct 22 23:55:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:26.902635 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb5a8b-435f-45c7-b1a5-16e9864afbc0 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:26.971208 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:27.010357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3253b6-1d82-47fe-805c-6091213b2811 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:27.023266 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb5a8b-435f-45c7-b1a5-16e9864afbc0 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:27.048000 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3573f83-1ee5-4445-b313-999812291cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:27.049121 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3573f83-1ee5-4445-b313-999812291cfb None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:27.049404 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3573f83-1ee5-4445-b313-999812291cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:27.049678 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3573f83-1ee5-4445-b313-999812291cfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:27.050432 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3573f83-1ee5-4445-b313-999812291cfb None admin] RBAC: Adding query filter params (role.id=159c6380cd984e998bb6a97ed7d98dac, user.id=5616f86f495348e6b7022d7ddb16df1c, scope.system=all) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:27.050832 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3573f83-1ee5-4445-b313-999812291cfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:27.052326 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3573f83-1ee5-4445-b313-999812291cfb None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:27.057632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 121/237] 10.222.0.13 () {56 vars in 1308 bytes} [Sat Oct 22 23:55:26 2022] GET /identity/v3/role_assignments?user.id=5616f86f495348e6b7022d7ddb16df1c&scope.system=all&role.id=159c6380cd984e998bb6a97ed7d98dac => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:27.062719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525db6c8-a945-460e-abc1-cc5c2cea0520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:27.063732 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525db6c8-a945-460e-abc1-cc5c2cea0520 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:27.064478 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525db6c8-a945-460e-abc1-cc5c2cea0520 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:27.064478 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525db6c8-a945-460e-abc1-cc5c2cea0520 None admin] PATH_INFO: `/v3/users/459e702ae60f4bc4a0ea0aa668e14160` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:27.065560 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525db6c8-a945-460e-abc1-cc5c2cea0520 None admin] RBAC: Authorizing `identity:get_user(user_id=459e702ae60f4bc4a0ea0aa668e14160)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:27.067307 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525db6c8-a945-460e-abc1-cc5c2cea0520 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:27.069038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 117/238] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:55:26 2022] GET /identity/v3/users/459e702ae60f4bc4a0ea0aa668e14160 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:55:27.075060 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3573f83-1ee5-4445-b313-999812291cfb None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:27.195645 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3573f83-1ee5-4445-b313-999812291cfb None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:27.227445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:27.233740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e8b569-0ec0-45e5-8535-30bb41f1ce09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:27.234402 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8b569-0ec0-45e5-8535-30bb41f1ce09 None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:27.234687 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8b569-0ec0-45e5-8535-30bb41f1ce09 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:27.235004 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e8b569-0ec0-45e5-8535-30bb41f1ce09 None admin] PATH_INFO: `/v3/system/users/459e702ae60f4bc4a0ea0aa668e14160/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:27.237178 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8b569-0ec0-45e5-8535-30bb41f1ce09 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=459e702ae60f4bc4a0ea0aa668e14160, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:27.238299 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e8b569-0ec0-45e5-8535-30bb41f1ce09 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:27.248378 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 122/239] 10.222.0.13 () {58 vars in 1334 bytes} [Sat Oct 22 23:55:27 2022] PUT /identity/v3/system/users/459e702ae60f4bc4a0ea0aa668e14160/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:55:29.107125 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b7ac4-c6fb-457b-8c72-8bec424a467c None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:29.107645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b7ac4-c6fb-457b-8c72-8bec424a467c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:29.107645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b7ac4-c6fb-457b-8c72-8bec424a467c None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:29.109302 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 118/240] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:29.115545 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a3dc5-95be-4deb-9761-62129598cda1 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:29.116047 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a3dc5-95be-4deb-9761-62129598cda1 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:29.116177 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a3dc5-95be-4deb-9761-62129598cda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:29.135593 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-7b1a3dc5-95be-4deb-9761-62129598cda1 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:29.136812 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-7b1a3dc5-95be-4deb-9761-62129598cda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:29.202658 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 123/241] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:29.215374 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e544fa-eedd-48ee-a52d-b77584240cfb None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:29.215663 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e544fa-eedd-48ee-a52d-b77584240cfb None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:29.215945 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e544fa-eedd-48ee-a52d-b77584240cfb None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:29.217320 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 119/242] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:29.222974 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e32df4-ce4c-4c78-a6f7-70cffc7390a3 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:29.223268 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e32df4-ce4c-4c78-a6f7-70cffc7390a3 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:29.223547 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e32df4-ce4c-4c78-a6f7-70cffc7390a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:29.242119 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-70e32df4-ce4c-4c78-a6f7-70cffc7390a3 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:29.243307 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-70e32df4-ce4c-4c78-a6f7-70cffc7390a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:29.317501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 124/243] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:29.383904 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842b290-69d2-4ee8-b41c-66972e89adb1 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:29.384107 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842b290-69d2-4ee8-b41c-66972e89adb1 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:29.384583 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842b290-69d2-4ee8-b41c-66972e89adb1 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:29.385926 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 120/244] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:29.391094 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fb965-d70a-4986-b97b-0a3f3b709310 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:29.391521 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fb965-d70a-4986-b97b-0a3f3b709310 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:29.391985 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fb965-d70a-4986-b97b-0a3f3b709310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:29.410377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-a89fb965-d70a-4986-b97b-0a3f3b709310 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:29.411542 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a89fb965-d70a-4986-b97b-0a3f3b709310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:29.462003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:29.474791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842b290-69d2-4ee8-b41c-66972e89adb1 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:29.483003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 125/245] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:29.489026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:29.582333 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842b290-69d2-4ee8-b41c-66972e89adb1 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:29.593060 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fb965-d70a-4986-b97b-0a3f3b709310 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:29.622130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc37b8b-8d8a-40fa-84fc-46f8cefb075f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:29.622974 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc37b8b-8d8a-40fa-84fc-46f8cefb075f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:29.623416 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc37b8b-8d8a-40fa-84fc-46f8cefb075f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:29.623852 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc37b8b-8d8a-40fa-84fc-46f8cefb075f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:29.625091 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc37b8b-8d8a-40fa-84fc-46f8cefb075f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:29.627220 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc37b8b-8d8a-40fa-84fc-46f8cefb075f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:29.628953 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 121/246] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:55:29.634818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc37b8b-8d8a-40fa-84fc-46f8cefb075f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:29.706549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fb965-d70a-4986-b97b-0a3f3b709310 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:29.721683 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:29.744213 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cd60d-4809-4d26-a43e-2304e20f837c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:29.744883 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cd60d-4809-4d26-a43e-2304e20f837c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:29.745175 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cd60d-4809-4d26-a43e-2304e20f837c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:29.745454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82cd60d-4809-4d26-a43e-2304e20f837c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:29.745752 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc37b8b-8d8a-40fa-84fc-46f8cefb075f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:29.757293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82cd60d-4809-4d26-a43e-2304e20f837c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:29.762182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-d82cd60d-4809-4d26-a43e-2304e20f837c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 22 23:55:29.762182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:29.762182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:29.762182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:29.762182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:29.762182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:29.762182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:29.762182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:29.762182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:29.762182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:29.762182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:29.762182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:29.762182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:29.762182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:29.762182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:29.762182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:29.763097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:29.763097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:29.763097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:29.763097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:29.763097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:29.763097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:29.763097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:29.763097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:29.763097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:29.763097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:29.763097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:29.763097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:29.763097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:29.763097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:29.763097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:29.763097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:29.763893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:29.763893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:29.763893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:29.763893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:29.763893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:29.763893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:29.763893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:29.763893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:29.763893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:29.763893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:29.763893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 22 23:55:29.763893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:29.763893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 126/247] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:55:29.769390 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cd60d-4809-4d26-a43e-2304e20f837c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:29.779596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93778bd-ad46-4fb9-bbdb-85881a48360b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:29.780204 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93778bd-ad46-4fb9-bbdb-85881a48360b None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:29.780509 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93778bd-ad46-4fb9-bbdb-85881a48360b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:29.780788 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93778bd-ad46-4fb9-bbdb-85881a48360b None admin] PATH_INFO: `/v3/groups` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:29.781417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93778bd-ad46-4fb9-bbdb-85881a48360b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:29.783646 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93778bd-ad46-4fb9-bbdb-85881a48360b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:29.798693 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 122/248] 10.222.0.13 () {60 vars in 1134 bytes} [Sat Oct 22 23:55:29 2022] POST /identity/v3/groups => generated 226 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 22 23:55:29.799666 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93778bd-ad46-4fb9-bbdb-85881a48360b None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:29.819444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:29.880186 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82cd60d-4809-4d26-a43e-2304e20f837c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:29.918589 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ca175-eff8-4899-a900-4ac94e2a0d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:29.919154 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ca175-eff8-4899-a900-4ac94e2a0d62 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:29.919441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ca175-eff8-4899-a900-4ac94e2a0d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:29.919720 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ca175-eff8-4899-a900-4ac94e2a0d62 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:29.929439 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93778bd-ad46-4fb9-bbdb-85881a48360b None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:29.932135 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ca175-eff8-4899-a900-4ac94e2a0d62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:29.937166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-0c9ca175-eff8-4899-a900-4ac94e2a0d62 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:29.937166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:29.937166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:29.937166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:29.937166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:29.937166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:29.937166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:29.937166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:29.937166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:29.937166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:29.937166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:29.937166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:29.937166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:29.937166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:29.937166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:29.937166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:29.938157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:29.938157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:29.938157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:29.938157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:29.938157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:29.938157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:29.938157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:29.938157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:29.938157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:29.938157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:29.938157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:29.938157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:29.938157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:29.938157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:29.938157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:29.938157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:29.939032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:29.939032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:29.939032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:29.939032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:29.939032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:29.939032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:29.939032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:29.939032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:29.939032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:29.939032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:29.939032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:29.939032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:29.939032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 127/249] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:29.945445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ca175-eff8-4899-a900-4ac94e2a0d62 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:29.971323 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770cb771-8ecd-4524-a56e-f7d4f1db3c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:29.972276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770cb771-8ecd-4524-a56e-f7d4f1db3c46 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:29.972759 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770cb771-8ecd-4524-a56e-f7d4f1db3c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:29.973239 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770cb771-8ecd-4524-a56e-f7d4f1db3c46 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:29.973979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770cb771-8ecd-4524-a56e-f7d4f1db3c46 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:29.974284 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770cb771-8ecd-4524-a56e-f7d4f1db3c46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:29.975670 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770cb771-8ecd-4524-a56e-f7d4f1db3c46 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:29.978443 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:29.992850 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 123/250] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Oct 22 23:55:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:29.996164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770cb771-8ecd-4524-a56e-f7d4f1db3c46 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:30.070590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ca175-eff8-4899-a900-4ac94e2a0d62 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:30.079529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:30.119400 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce03bac-d191-40a0-bc2a-05895cc639aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:30.120097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce03bac-d191-40a0-bc2a-05895cc639aa None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:30.120294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce03bac-d191-40a0-bc2a-05895cc639aa None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:30.120603 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce03bac-d191-40a0-bc2a-05895cc639aa None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:30.121270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce03bac-d191-40a0-bc2a-05895cc639aa None admin] RBAC: Adding query filter params (name=member) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:30.121576 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce03bac-d191-40a0-bc2a-05895cc639aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:30.124090 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce03bac-d191-40a0-bc2a-05895cc639aa None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:30.136289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770cb771-8ecd-4524-a56e-f7d4f1db3c46 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:30.136815 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 128/251] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Oct 22 23:55:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:55:30.145744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce03bac-d191-40a0-bc2a-05895cc639aa None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:30.182932 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6c1a5-7a32-4b14-b878-38232a674d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:30.183497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6c1a5-7a32-4b14-b878-38232a674d5f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:30.183784 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6c1a5-7a32-4b14-b878-38232a674d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:30.184067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6c1a5-7a32-4b14-b878-38232a674d5f None admin] PATH_INFO: `/v3/users/5616f86f495348e6b7022d7ddb16df1c` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:30.187308 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f6c1a5-7a32-4b14-b878-38232a674d5f None admin] RBAC: Authorizing `identity:get_user(user_id=5616f86f495348e6b7022d7ddb16df1c)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:30.189424 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f6c1a5-7a32-4b14-b878-38232a674d5f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:30.191638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 124/252] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:55:29 2022] GET /identity/v3/users/5616f86f495348e6b7022d7ddb16df1c => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:55:30.196873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6c1a5-7a32-4b14-b878-38232a674d5f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:30.281507 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce03bac-d191-40a0-bc2a-05895cc639aa None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:30.284432 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:30.329210 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c118a-216f-4b5d-8f89-b2ab82e24ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:30.329891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c118a-216f-4b5d-8f89-b2ab82e24ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:30.330175 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c118a-216f-4b5d-8f89-b2ab82e24ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:30.330453 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c118a-216f-4b5d-8f89-b2ab82e24ab0 None admin] PATH_INFO: `/v3/users/459e702ae60f4bc4a0ea0aa668e14160` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:30.330824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6c1a5-7a32-4b14-b878-38232a674d5f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:30.331604 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0c118a-216f-4b5d-8f89-b2ab82e24ab0 None admin] RBAC: Authorizing `identity:get_user(user_id=459e702ae60f4bc4a0ea0aa668e14160)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:30.334094 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0c118a-216f-4b5d-8f89-b2ab82e24ab0 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:30.334954 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 129/253] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:55:30 2022] GET /identity/v3/users/459e702ae60f4bc4a0ea0aa668e14160 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:55:30.341340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c118a-216f-4b5d-8f89-b2ab82e24ab0 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:30.383464 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e629b-c33b-4477-bd7f-1a7f003779bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:30.384260 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e629b-c33b-4477-bd7f-1a7f003779bb None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:30.384570 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e629b-c33b-4477-bd7f-1a7f003779bb None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:30.384854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e629b-c33b-4477-bd7f-1a7f003779bb None admin] PATH_INFO: `/v3/system/users/5616f86f495348e6b7022d7ddb16df1c/roles/159c6380cd984e998bb6a97ed7d98dac` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:30.386468 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e629b-c33b-4477-bd7f-1a7f003779bb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5616f86f495348e6b7022d7ddb16df1c, role_id=159c6380cd984e998bb6a97ed7d98dac)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:30.388085 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:30.388933 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e629b-c33b-4477-bd7f-1a7f003779bb None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:30.406131 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 125/254] 10.222.0.13 () {58 vars in 1334 bytes} [Sat Oct 22 23:55:30 2022] PUT /identity/v3/system/users/5616f86f495348e6b7022d7ddb16df1c/roles/159c6380cd984e998bb6a97ed7d98dac => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:55:30.473622 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c118a-216f-4b5d-8f89-b2ab82e24ab0 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:30.513612 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d05acc-d156-4e11-90ad-b0335b538765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:30.514236 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d05acc-d156-4e11-90ad-b0335b538765 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:30.514514 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d05acc-d156-4e11-90ad-b0335b538765 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:30.514790 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d05acc-d156-4e11-90ad-b0335b538765 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:30.515580 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d05acc-d156-4e11-90ad-b0335b538765 None admin] RBAC: Adding query filter params (role.id=4bd2874c95634a50aac9fccbddc7c674, user.id=459e702ae60f4bc4a0ea0aa668e14160, scope.system=all) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:30.515891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d05acc-d156-4e11-90ad-b0335b538765 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:30.518487 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d05acc-d156-4e11-90ad-b0335b538765 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:30.533128 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 130/255] 10.222.0.13 () {56 vars in 1308 bytes} [Sat Oct 22 23:55:30 2022] GET /identity/v3/role_assignments?user.id=459e702ae60f4bc4a0ea0aa668e14160&scope.system=all&role.id=4bd2874c95634a50aac9fccbddc7c674 => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:55:32.506690 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd68f9d-1c27-4448-8602-54d9b1a8a576 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:32.507622 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd68f9d-1c27-4448-8602-54d9b1a8a576 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:32.508105 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd68f9d-1c27-4448-8602-54d9b1a8a576 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:32.509685 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 126/256] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:32.520604 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacaced5-1018-4a66-b518-08f3c1c79212 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:32.522464 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580cfd2-ceb4-4615-a566-353fc6b296be None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:32.522747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580cfd2-ceb4-4615-a566-353fc6b296be None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:32.523029 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9580cfd2-ceb4-4615-a566-353fc6b296be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:32.524905 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacaced5-1018-4a66-b518-08f3c1c79212 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:32.525394 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacaced5-1018-4a66-b518-08f3c1c79212 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:32.526983 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 131/257] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:32.533255 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de57037-5b58-4256-9351-76c3dd30de50 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:32.533833 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de57037-5b58-4256-9351-76c3dd30de50 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:32.534313 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de57037-5b58-4256-9351-76c3dd30de50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:32.549332 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-9580cfd2-ceb4-4615-a566-353fc6b296be None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:32.550744 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9580cfd2-ceb4-4615-a566-353fc6b296be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:32.569590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-0de57037-5b58-4256-9351-76c3dd30de50 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:32.571135 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-0de57037-5b58-4256-9351-76c3dd30de50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:32.646471 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 127/258] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 22 23:55:32.667304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 132/259] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 22 23:55:32.972336 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580cfd2-ceb4-4615-a566-353fc6b296be None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:32.981716 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9580cfd2-ceb4-4615-a566-353fc6b296be None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:32.988213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f040d-08a2-4f04-83d0-742126aeb3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:32.989124 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f040d-08a2-4f04-83d0-742126aeb3e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:32.989603 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f040d-08a2-4f04-83d0-742126aeb3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:32.990072 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f040d-08a2-4f04-83d0-742126aeb3e0 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:33.001855 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82f040d-08a2-4f04-83d0-742126aeb3e0 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:33.007013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-c82f040d-08a2-4f04-83d0-742126aeb3e0 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:33.007013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:33.007013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:33.007013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:33.007013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:33.007013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:33.007013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:33.007013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:33.007013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:33.007013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:33.007013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:33.007013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:33.007013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:33.007013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:33.007013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:33.007013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:33.007906 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:33.007906 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:33.007906 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:33.007906 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:33.007906 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:33.007906 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:33.007906 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:33.007906 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:33.007906 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:33.007906 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:33.007906 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:33.007906 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:33.007906 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:33.007906 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:33.007906 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:33.007906 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:33.008737 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:33.008737 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:33.008737 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:33.008737 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:33.008737 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:33.008737 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:33.008737 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:33.008737 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:33.008737 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:33.008737 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:33.008737 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:33.008737 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:33.010958 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 128/260] 10.222.0.13 () {56 vars in 1120 bytes} [Sat Oct 22 23:55:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 39 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:33.017656 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de57037-5b58-4256-9351-76c3dd30de50 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:33.027540 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de57037-5b58-4256-9351-76c3dd30de50 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:33.032266 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6d54e-89c5-46eb-8772-ee1c5794b5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:33.033123 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6d54e-89c5-46eb-8772-ee1c5794b5bc admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:33.033601 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6d54e-89c5-46eb-8772-ee1c5794b5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:33.034067 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6d54e-89c5-46eb-8772-ee1c5794b5bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:33.034923 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf6d54e-89c5-46eb-8772-ee1c5794b5bc admin admin] RBAC: Adding query filter params (name=member) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:33.035435 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf6d54e-89c5-46eb-8772-ee1c5794b5bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:33.037423 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: WARNING py.warnings [None req-adf6d54e-89c5-46eb-8772-ee1c5794b5bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 22 23:55:33.037423 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: warnings.warn(msg) Oct 22 23:55:33.037423 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: Oct 22 23:55:33.038665 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf6d54e-89c5-46eb-8772-ee1c5794b5bc admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:33.051637 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 133/261] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Oct 22 23:55:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:33.057437 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f040d-08a2-4f04-83d0-742126aeb3e0 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:33.074969 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f040d-08a2-4f04-83d0-742126aeb3e0 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:33.080113 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0586d8-9086-4b60-8088-3a44c1636383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:33.081021 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0586d8-9086-4b60-8088-3a44c1636383 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:33.081515 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0586d8-9086-4b60-8088-3a44c1636383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:33.081993 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0586d8-9086-4b60-8088-3a44c1636383 admin admin] PATH_INFO: `/v3/groups/8a1180354f9445f49e7ad23bac3f4557` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:33.095434 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0586d8-9086-4b60-8088-3a44c1636383 admin admin] RBAC: Authorizing `identity:get_group(group_id=8a1180354f9445f49e7ad23bac3f4557)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:33.097584 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: WARNING py.warnings [None req-fe0586d8-9086-4b60-8088-3a44c1636383 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 22 23:55:33.097584 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: warnings.warn(msg) Oct 22 23:55:33.097584 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: Oct 22 23:55:33.099156 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0586d8-9086-4b60-8088-3a44c1636383 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:33.101355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 129/262] 10.222.0.13 () {56 vars in 1201 bytes} [Sat Oct 22 23:55:33 2022] GET /identity/v3/groups/8a1180354f9445f49e7ad23bac3f4557 => generated 226 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:33.107109 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6d54e-89c5-46eb-8772-ee1c5794b5bc admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:33.121539 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6d54e-89c5-46eb-8772-ee1c5794b5bc admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:33.129357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa5d4d-bd32-4430-be50-48ca45a078fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:33.129992 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa5d4d-bd32-4430-be50-48ca45a078fe admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:33.130274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa5d4d-bd32-4430-be50-48ca45a078fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:33.130545 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fa5d4d-bd32-4430-be50-48ca45a078fe admin admin] PATH_INFO: `/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:33.137276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa5d4d-bd32-4430-be50-48ca45a078fe admin admin] RBAC: Authorizing `identity:get_project(project_id=4d6b66c470f74a4dac6f66e6fcee2a51)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:33.139418 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fa5d4d-bd32-4430-be50-48ca45a078fe admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:33.141663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 134/263] 10.222.0.13 () {56 vars in 1207 bytes} [Sat Oct 22 23:55:33 2022] GET /identity/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51 => generated 298 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:33.147364 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0586d8-9086-4b60-8088-3a44c1636383 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:33.162244 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0586d8-9086-4b60-8088-3a44c1636383 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:33.167861 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a63fa-dc88-4dbd-b730-8fe6563805d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:33.168571 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a63fa-dc88-4dbd-b730-8fe6563805d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:33.168862 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a63fa-dc88-4dbd-b730-8fe6563805d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:33.169141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a63fa-dc88-4dbd-b730-8fe6563805d0 admin admin] PATH_INFO: `/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51/groups/8a1180354f9445f49e7ad23bac3f4557/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:33.173202 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a63fa-dc88-4dbd-b730-8fe6563805d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d6b66c470f74a4dac6f66e6fcee2a51, group_id=8a1180354f9445f49e7ad23bac3f4557, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:33.174307 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: WARNING py.warnings [None req-e53a63fa-dc88-4dbd-b730-8fe6563805d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 22 23:55:33.174307 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: warnings.warn(msg) Oct 22 23:55:33.174307 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: Oct 22 23:55:33.176275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a63fa-dc88-4dbd-b730-8fe6563805d0 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:33.196584 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 130/264] 10.222.0.13 () {58 vars in 1463 bytes} [Sat Oct 22 23:55:33 2022] PUT /identity/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51/groups/8a1180354f9445f49e7ad23bac3f4557/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:55:33.267765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa5d4d-bd32-4430-be50-48ca45a078fe admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:33.278459 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fa5d4d-bd32-4430-be50-48ca45a078fe admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:33.315089 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:33.358494 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e31219-a371-467a-8f90-7b20afb60539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:33.359239 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e31219-a371-467a-8f90-7b20afb60539 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:33.359545 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e31219-a371-467a-8f90-7b20afb60539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:33.359992 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e31219-a371-467a-8f90-7b20afb60539 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:33.378546 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e31219-a371-467a-8f90-7b20afb60539 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:33.383605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-a3e31219-a371-467a-8f90-7b20afb60539 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 22 23:55:33.383605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:33.383605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:33.383605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:33.383605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:33.383605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:33.383605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:33.383605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:33.383605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:33.383605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:33.383605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:33.383605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:33.383605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:33.383605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:33.383605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:33.383605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:33.385776 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:33.385776 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:33.385776 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:33.385776 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:33.385776 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:33.385776 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:33.385776 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:33.385776 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:33.385776 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:33.385776 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:33.385776 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:33.385776 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:33.385776 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:33.385776 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:33.385776 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:33.385776 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:33.387688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:33.387688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:33.387688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:33.387688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:33.387688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:33.387688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:33.387688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:33.387688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:33.387688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:33.387688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:33.387688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 22 23:55:33.387688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:33.387688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 135/265] 10.222.0.13 () {56 vars in 1135 bytes} [Sat Oct 22 23:55:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 119 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:33.391603 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a63fa-dc88-4dbd-b730-8fe6563805d0 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:33.411072 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a63fa-dc88-4dbd-b730-8fe6563805d0 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:33.430402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79a08f-25e1-48e3-9df9-55284afa4d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:33.431034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79a08f-25e1-48e3-9df9-55284afa4d2b admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:33.431314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79a08f-25e1-48e3-9df9-55284afa4d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:33.431586 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79a08f-25e1-48e3-9df9-55284afa4d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:33.432256 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79a08f-25e1-48e3-9df9-55284afa4d2b admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:33.432581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79a08f-25e1-48e3-9df9-55284afa4d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:33.433600 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: WARNING py.warnings [None req-af79a08f-25e1-48e3-9df9-55284afa4d2b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 22 23:55:33.433600 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: warnings.warn(msg) Oct 22 23:55:33.433600 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: Oct 22 23:55:33.434491 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79a08f-25e1-48e3-9df9-55284afa4d2b admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:33.451694 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 131/266] 10.222.0.13 () {56 vars in 1132 bytes} [Sat Oct 22 23:55:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:33.458479 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e31219-a371-467a-8f90-7b20afb60539 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:33.474090 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e31219-a371-467a-8f90-7b20afb60539 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:33.485434 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776222bc-6da3-4f25-9015-d75f9de16220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:33.486410 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776222bc-6da3-4f25-9015-d75f9de16220 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:33.486410 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776222bc-6da3-4f25-9015-d75f9de16220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:33.486736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776222bc-6da3-4f25-9015-d75f9de16220 admin admin] PATH_INFO: `/v3/groups/8a1180354f9445f49e7ad23bac3f4557` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:33.489743 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776222bc-6da3-4f25-9015-d75f9de16220 admin admin] RBAC: Authorizing `identity:get_group(group_id=8a1180354f9445f49e7ad23bac3f4557)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:33.490818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: WARNING py.warnings [None req-776222bc-6da3-4f25-9015-d75f9de16220 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 22 23:55:33.490818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: warnings.warn(msg) Oct 22 23:55:33.490818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: Oct 22 23:55:33.492155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776222bc-6da3-4f25-9015-d75f9de16220 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:33.494374 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 136/267] 10.222.0.13 () {56 vars in 1201 bytes} [Sat Oct 22 23:55:33 2022] GET /identity/v3/groups/8a1180354f9445f49e7ad23bac3f4557 => generated 226 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:33.500326 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79a08f-25e1-48e3-9df9-55284afa4d2b admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:33.517242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79a08f-25e1-48e3-9df9-55284afa4d2b admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:33.524901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8976c-7a3d-4df1-9ea7-2c21843acc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:33.525484 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8976c-7a3d-4df1-9ea7-2c21843acc9b admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:33.525763 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8976c-7a3d-4df1-9ea7-2c21843acc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:33.526035 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8976c-7a3d-4df1-9ea7-2c21843acc9b admin admin] PATH_INFO: `/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:33.529154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8976c-7a3d-4df1-9ea7-2c21843acc9b admin admin] RBAC: Authorizing `identity:get_project(project_id=4d6b66c470f74a4dac6f66e6fcee2a51)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:33.531279 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b8976c-7a3d-4df1-9ea7-2c21843acc9b admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:33.533534 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 132/268] 10.222.0.13 () {56 vars in 1207 bytes} [Sat Oct 22 23:55:33 2022] GET /identity/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51 => generated 298 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:33.539410 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776222bc-6da3-4f25-9015-d75f9de16220 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:33.552370 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776222bc-6da3-4f25-9015-d75f9de16220 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:33.559095 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487e4ff-cd8e-417e-95c1-51c24f785655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:33.559781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487e4ff-cd8e-417e-95c1-51c24f785655 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:33.560063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487e4ff-cd8e-417e-95c1-51c24f785655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:33.560343 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487e4ff-cd8e-417e-95c1-51c24f785655 admin admin] PATH_INFO: `/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51/groups/8a1180354f9445f49e7ad23bac3f4557/roles/cf8116e4e82748ac872d8763d113ba61` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:33.566860 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f487e4ff-cd8e-417e-95c1-51c24f785655 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d6b66c470f74a4dac6f66e6fcee2a51, group_id=8a1180354f9445f49e7ad23bac3f4557, role_id=cf8116e4e82748ac872d8763d113ba61)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:33.567927 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: WARNING py.warnings [None req-f487e4ff-cd8e-417e-95c1-51c24f785655 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 22 23:55:33.567927 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: warnings.warn(msg) Oct 22 23:55:33.567927 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: Oct 22 23:55:33.570297 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f487e4ff-cd8e-417e-95c1-51c24f785655 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:33.590608 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 137/269] 10.222.0.13 () {58 vars in 1463 bytes} [Sat Oct 22 23:55:33 2022] PUT /identity/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51/groups/8a1180354f9445f49e7ad23bac3f4557/roles/cf8116e4e82748ac872d8763d113ba61 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:55:33.711767 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562c53d-bdd5-4014-a83f-9c4bec3366e1 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:33.712439 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562c53d-bdd5-4014-a83f-9c4bec3366e1 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:33.713003 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562c53d-bdd5-4014-a83f-9c4bec3366e1 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:33.714700 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 133/270] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:33.720555 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f10b7-7acf-49d2-951e-20147dcec99b None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:33.720852 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f10b7-7acf-49d2-951e-20147dcec99b None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:33.721133 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f10b7-7acf-49d2-951e-20147dcec99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:33.747657 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-a97f10b7-7acf-49d2-951e-20147dcec99b None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:33.748941 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a97f10b7-7acf-49d2-951e-20147dcec99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:33.840540 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 138/271] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:33 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 22 23:55:33.850030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167119d4-b236-4232-ac9f-d501065d880b None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:33.850687 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167119d4-b236-4232-ac9f-d501065d880b None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:33.851155 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167119d4-b236-4232-ac9f-d501065d880b None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:33.852779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 134/272] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:33.862231 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9c05a-394f-42f2-8f07-f5fa62821527 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:33.862520 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9c05a-394f-42f2-8f07-f5fa62821527 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:33.862799 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9c05a-394f-42f2-8f07-f5fa62821527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:33.884286 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-69e9c05a-394f-42f2-8f07-f5fa62821527 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:33.885529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-69e9c05a-394f-42f2-8f07-f5fa62821527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:33.972704 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 139/273] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:34.135120 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416c047-8c7f-48cb-a242-1bfd7ef53b17 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:34.135451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416c047-8c7f-48cb-a242-1bfd7ef53b17 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:34.135691 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416c047-8c7f-48cb-a242-1bfd7ef53b17 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:34.137381 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 135/274] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:34.145696 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fbee9-4113-49a5-9f8f-c5917dbd1776 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:34.146262 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fbee9-4113-49a5-9f8f-c5917dbd1776 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:34.146758 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fbee9-4113-49a5-9f8f-c5917dbd1776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:34.173911 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-283fbee9-4113-49a5-9f8f-c5917dbd1776 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:34.175447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-283fbee9-4113-49a5-9f8f-c5917dbd1776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:34.185482 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 140/275] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 22 23:55:34.213327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416c047-8c7f-48cb-a242-1bfd7ef53b17 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:34.233823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416c047-8c7f-48cb-a242-1bfd7ef53b17 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:34.243751 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044e55c-e1da-4a91-a49c-1bc8a09354f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:34.244382 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044e55c-e1da-4a91-a49c-1bc8a09354f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:34.244691 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044e55c-e1da-4a91-a49c-1bc8a09354f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:34.244970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044e55c-e1da-4a91-a49c-1bc8a09354f7 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:34.259014 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c044e55c-e1da-4a91-a49c-1bc8a09354f7 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:34.264213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-c044e55c-e1da-4a91-a49c-1bc8a09354f7 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:34.264213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:34.264213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:34.264213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:34.264213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:34.264213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:34.264213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:34.264213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:34.264213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:34.264213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:34.264213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:34.264213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:34.264213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:34.264213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:34.264213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:34.264213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:34.265132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:34.265132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:34.265132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:34.265132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:34.265132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:34.265132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:34.265132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:34.265132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:34.265132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:34.265132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:34.265132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:34.265132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:34.265132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:34.265132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:34.265132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:34.265132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:34.265954 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:34.265954 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:34.265954 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:34.265954 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:34.265954 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:34.265954 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:34.265954 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:34.265954 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:34.265954 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:34.265954 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:34.265954 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:34.265954 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:34.268892 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 136/276] 10.222.0.13 () {56 vars in 1120 bytes} [Sat Oct 22 23:55:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 56 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:34.274754 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283fbee9-4113-49a5-9f8f-c5917dbd1776 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:34.285534 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283fbee9-4113-49a5-9f8f-c5917dbd1776 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:34.296014 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35e211-68b2-466a-b5dd-a077cf55f8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:34.296878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35e211-68b2-466a-b5dd-a077cf55f8df admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:34.297879 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35e211-68b2-466a-b5dd-a077cf55f8df admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:34.298356 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35e211-68b2-466a-b5dd-a077cf55f8df admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:34.299300 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae35e211-68b2-466a-b5dd-a077cf55f8df admin admin] RBAC: Adding query filter params (name=member) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:34.299820 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae35e211-68b2-466a-b5dd-a077cf55f8df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:34.301660 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae35e211-68b2-466a-b5dd-a077cf55f8df admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:34.311072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 141/277] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Oct 22 23:55:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:34.317213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044e55c-e1da-4a91-a49c-1bc8a09354f7 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:34.327514 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044e55c-e1da-4a91-a49c-1bc8a09354f7 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:34.336565 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea05cc-ca4c-4d55-b729-e4062b8ec0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:34.337450 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea05cc-ca4c-4d55-b729-e4062b8ec0d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:34.337952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea05cc-ca4c-4d55-b729-e4062b8ec0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:34.338423 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea05cc-ca4c-4d55-b729-e4062b8ec0d8 admin admin] PATH_INFO: `/v3/groups/8a1180354f9445f49e7ad23bac3f4557` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:34.339780 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea05cc-ca4c-4d55-b729-e4062b8ec0d8 admin admin] RBAC: Authorizing `identity:get_group(group_id=8a1180354f9445f49e7ad23bac3f4557)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:34.341938 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea05cc-ca4c-4d55-b729-e4062b8ec0d8 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:34.343697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 137/278] 10.222.0.13 () {56 vars in 1201 bytes} [Sat Oct 22 23:55:34 2022] GET /identity/v3/groups/8a1180354f9445f49e7ad23bac3f4557 => generated 226 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:34.349505 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35e211-68b2-466a-b5dd-a077cf55f8df admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:34.359830 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35e211-68b2-466a-b5dd-a077cf55f8df admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:34.368157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b108ff1-6667-4864-b13b-cd1398cf14e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:34.369072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b108ff1-6667-4864-b13b-cd1398cf14e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:34.369558 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b108ff1-6667-4864-b13b-cd1398cf14e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:34.370047 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b108ff1-6667-4864-b13b-cd1398cf14e7 admin admin] PATH_INFO: `/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:34.371394 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b108ff1-6667-4864-b13b-cd1398cf14e7 admin admin] RBAC: Authorizing `identity:get_project(project_id=79baed4da64f4fc2b256aa6fc529fbd2)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:34.373671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b108ff1-6667-4864-b13b-cd1398cf14e7 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:34.375531 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 142/279] 10.222.0.13 () {56 vars in 1207 bytes} [Sat Oct 22 23:55:34 2022] GET /identity/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2 => generated 302 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:34.381877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea05cc-ca4c-4d55-b729-e4062b8ec0d8 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:34.395967 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea05cc-ca4c-4d55-b729-e4062b8ec0d8 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:34.400823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b47e8-38d3-47f5-a489-c86861ca1a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:34.401711 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b47e8-38d3-47f5-a489-c86861ca1a0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:34.402193 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b47e8-38d3-47f5-a489-c86861ca1a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:34.402702 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b47e8-38d3-47f5-a489-c86861ca1a0a admin admin] PATH_INFO: `/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2/groups/8a1180354f9445f49e7ad23bac3f4557/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:34.404828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b47e8-38d3-47f5-a489-c86861ca1a0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=79baed4da64f4fc2b256aa6fc529fbd2, group_id=8a1180354f9445f49e7ad23bac3f4557, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:34.407880 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b47e8-38d3-47f5-a489-c86861ca1a0a admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:34.424028 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 138/280] 10.222.0.13 () {58 vars in 1463 bytes} [Sat Oct 22 23:55:34 2022] PUT /identity/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2/groups/8a1180354f9445f49e7ad23bac3f4557/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:55:34.539464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b108ff1-6667-4864-b13b-cd1398cf14e7 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:34.550694 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b108ff1-6667-4864-b13b-cd1398cf14e7 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:34.554725 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:34.591065 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b47e8-38d3-47f5-a489-c86861ca1a0a admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:34.598251 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:34.615141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa12b2c-5680-4ffa-9d1e-0c740a847109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:34.618114 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa12b2c-5680-4ffa-9d1e-0c740a847109 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:34.618114 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa12b2c-5680-4ffa-9d1e-0c740a847109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:34.618114 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa12b2c-5680-4ffa-9d1e-0c740a847109 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:34.630570 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa12b2c-5680-4ffa-9d1e-0c740a847109 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:34.635950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-bfa12b2c-5680-4ffa-9d1e-0c740a847109 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 22 23:55:34.635950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:34.635950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:34.635950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:34.635950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:34.635950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:34.635950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:34.635950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:34.635950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:34.635950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:34.635950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:34.635950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:34.635950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:34.635950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:34.635950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:34.635950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:34.637079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:34.637079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:34.637079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:34.637079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:34.637079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:34.637079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:34.637079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:34.637079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:34.637079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:34.637079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:34.637079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:34.637079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:34.637079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:34.637079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:34.637079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:34.637079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:34.637874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:34.637874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:34.637874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:34.637874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:34.637874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:34.637874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:34.637874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:34.637874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:34.637874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:34.637874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:34.637874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 22 23:55:34.637874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:34.642494 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 143/281] 10.222.0.13 () {56 vars in 1135 bytes} [Sat Oct 22 23:55:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 98 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:34.643818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa12b2c-5680-4ffa-9d1e-0c740a847109 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:34.654632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa12b2c-5680-4ffa-9d1e-0c740a847109 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:34.664377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb22c1-4f60-4d58-a535-aa074e8a1bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:34.664377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb22c1-4f60-4d58-a535-aa074e8a1bdb admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:34.664377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb22c1-4f60-4d58-a535-aa074e8a1bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:34.664377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb22c1-4f60-4d58-a535-aa074e8a1bdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:34.664887 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb22c1-4f60-4d58-a535-aa074e8a1bdb admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:34.664887 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb22c1-4f60-4d58-a535-aa074e8a1bdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:34.664887 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb22c1-4f60-4d58-a535-aa074e8a1bdb admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:34.678992 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 144/282] 10.222.0.13 () {56 vars in 1132 bytes} [Sat Oct 22 23:55:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:55:34.678992 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb22c1-4f60-4d58-a535-aa074e8a1bdb admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:34.689455 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb22c1-4f60-4d58-a535-aa074e8a1bdb admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:34.700964 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05509a9c-26ce-45b9-abfd-a277f39a3d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:34.701616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05509a9c-26ce-45b9-abfd-a277f39a3d82 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:34.701901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05509a9c-26ce-45b9-abfd-a277f39a3d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:34.702177 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05509a9c-26ce-45b9-abfd-a277f39a3d82 admin admin] PATH_INFO: `/v3/groups/8a1180354f9445f49e7ad23bac3f4557` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:34.703443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05509a9c-26ce-45b9-abfd-a277f39a3d82 admin admin] RBAC: Authorizing `identity:get_group(group_id=8a1180354f9445f49e7ad23bac3f4557)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:34.705969 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05509a9c-26ce-45b9-abfd-a277f39a3d82 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:34.713805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 145/283] 10.222.0.13 () {56 vars in 1201 bytes} [Sat Oct 22 23:55:34 2022] GET /identity/v3/groups/8a1180354f9445f49e7ad23bac3f4557 => generated 226 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:34.721619 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05509a9c-26ce-45b9-abfd-a277f39a3d82 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:34.739131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05509a9c-26ce-45b9-abfd-a277f39a3d82 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:34.751274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9edcc-6090-4c54-aa72-b1bef5bfd84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:34.751932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9edcc-6090-4c54-aa72-b1bef5bfd84a admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:34.752219 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9edcc-6090-4c54-aa72-b1bef5bfd84a admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:34.752517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9edcc-6090-4c54-aa72-b1bef5bfd84a admin admin] PATH_INFO: `/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:34.753873 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9edcc-6090-4c54-aa72-b1bef5bfd84a admin admin] RBAC: Authorizing `identity:get_project(project_id=79baed4da64f4fc2b256aa6fc529fbd2)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:34.755953 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9edcc-6090-4c54-aa72-b1bef5bfd84a admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:34.757711 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 146/284] 10.222.0.13 () {56 vars in 1207 bytes} [Sat Oct 22 23:55:34 2022] GET /identity/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2 => generated 302 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:55:34.765546 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9edcc-6090-4c54-aa72-b1bef5bfd84a admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:34.782913 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9edcc-6090-4c54-aa72-b1bef5bfd84a admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:34.785444 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b47e8-38d3-47f5-a489-c86861ca1a0a admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:34.799598 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75c05f-1c87-47da-a69c-1c820893b6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:34.800283 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75c05f-1c87-47da-a69c-1c820893b6e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:34.800601 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75c05f-1c87-47da-a69c-1c820893b6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:34.800879 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75c05f-1c87-47da-a69c-1c820893b6e9 admin admin] PATH_INFO: `/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2/groups/8a1180354f9445f49e7ad23bac3f4557/roles/cf8116e4e82748ac872d8763d113ba61` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:34.806937 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c75c05f-1c87-47da-a69c-1c820893b6e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79baed4da64f4fc2b256aa6fc529fbd2, group_id=8a1180354f9445f49e7ad23bac3f4557, role_id=cf8116e4e82748ac872d8763d113ba61)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:34.810139 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c75c05f-1c87-47da-a69c-1c820893b6e9 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:34.835641 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 147/285] 10.222.0.13 () {58 vars in 1463 bytes} [Sat Oct 22 23:55:34 2022] PUT /identity/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2/groups/8a1180354f9445f49e7ad23bac3f4557/roles/cf8116e4e82748ac872d8763d113ba61 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:55:34.838245 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1653e-4561-4a55-98a2-b93b8b0f2dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:34.839260 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1653e-4561-4a55-98a2-b93b8b0f2dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:34.839874 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1653e-4561-4a55-98a2-b93b8b0f2dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:34.840349 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1653e-4561-4a55-98a2-b93b8b0f2dc5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:34.857123 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1653e-4561-4a55-98a2-b93b8b0f2dc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:34.863338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-c1e1653e-4561-4a55-98a2-b93b8b0f2dc5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 22 23:55:34.863338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:34.863338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:34.863338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:34.863338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:34.863338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:34.863338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:34.863338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:34.863338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:34.863338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:34.863338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:34.863338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:34.863338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:34.863338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:34.863338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:34.863338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:34.864263 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:34.864263 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:34.864263 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:34.864263 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:34.864263 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:34.864263 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:34.864263 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:34.864263 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:34.864263 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:34.864263 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:34.864263 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:34.864263 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:34.864263 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:34.864263 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:34.864263 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:34.864263 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:34.865210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:34.865210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:34.865210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:34.865210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:34.865210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:34.865210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:34.865210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:34.865210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:34.865210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:34.865210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:34.865210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 22 23:55:34.865210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:34.867363 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 139/286] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:55:34.873886 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75c05f-1c87-47da-a69c-1c820893b6e9 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:34.996878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75c05f-1c87-47da-a69c-1c820893b6e9 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:35.017641 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:35.044282 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c171ba-0d60-4dac-9535-a941eaf76ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:35.044949 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c171ba-0d60-4dac-9535-a941eaf76ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:35.045237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c171ba-0d60-4dac-9535-a941eaf76ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:35.045554 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c171ba-0d60-4dac-9535-a941eaf76ad7 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:35.046240 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c171ba-0d60-4dac-9535-a941eaf76ad7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:35.046548 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c171ba-0d60-4dac-9535-a941eaf76ad7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:35.048519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c171ba-0d60-4dac-9535-a941eaf76ad7 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:35.062834 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 148/287] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Oct 22 23:55:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:35.064844 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1653e-4561-4a55-98a2-b93b8b0f2dc5 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:35.188814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:35.217806 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965473ed-1c3e-4040-b3ad-f1b4887bd0e7 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:35.218103 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965473ed-1c3e-4040-b3ad-f1b4887bd0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:35.218383 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965473ed-1c3e-4040-b3ad-f1b4887bd0e7 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:35.220117 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 149/288] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 22 23:55:35.226702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573d068-c419-4784-be20-8cfe03d7a5da None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:35.226999 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573d068-c419-4784-be20-8cfe03d7a5da None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:35.227276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573d068-c419-4784-be20-8cfe03d7a5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:35.235481 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1653e-4561-4a55-98a2-b93b8b0f2dc5 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:35.265653 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-0573d068-c419-4784-be20-8cfe03d7a5da None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:35.266928 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-0573d068-c419-4784-be20-8cfe03d7a5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:35.286219 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac636f-c42b-4eb9-b326-12bf024be092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:35.287480 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac636f-c42b-4eb9-b326-12bf024be092 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:35.287944 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac636f-c42b-4eb9-b326-12bf024be092 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:35.288392 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac636f-c42b-4eb9-b326-12bf024be092 None admin] PATH_INFO: `/v3/users/5616f86f495348e6b7022d7ddb16df1c` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:35.289832 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac636f-c42b-4eb9-b326-12bf024be092 None admin] RBAC: Authorizing `identity:get_user(user_id=5616f86f495348e6b7022d7ddb16df1c)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:35.291569 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac636f-c42b-4eb9-b326-12bf024be092 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:35.294126 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 140/289] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:55:35 2022] GET /identity/v3/users/5616f86f495348e6b7022d7ddb16df1c => generated 293 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:55:35.300200 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac636f-c42b-4eb9-b326-12bf024be092 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:35.391815 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 150/290] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 22 23:55:35.433291 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac636f-c42b-4eb9-b326-12bf024be092 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:35.466484 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:35.478137 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96f9ab-7c61-48e8-887c-47ffc27369ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:35.479031 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96f9ab-7c61-48e8-887c-47ffc27369ba None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:35.479484 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96f9ab-7c61-48e8-887c-47ffc27369ba None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:35.479929 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96f9ab-7c61-48e8-887c-47ffc27369ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:35.480841 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96f9ab-7c61-48e8-887c-47ffc27369ba None admin] RBAC: Adding query filter params (role.id=159c6380cd984e998bb6a97ed7d98dac, user.id=5616f86f495348e6b7022d7ddb16df1c, scope.system=all) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:35.481344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96f9ab-7c61-48e8-887c-47ffc27369ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:35.482942 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96f9ab-7c61-48e8-887c-47ffc27369ba None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:35.501195 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 141/291] 10.222.0.13 () {56 vars in 1308 bytes} [Sat Oct 22 23:55:35 2022] GET /identity/v3/role_assignments?user.id=5616f86f495348e6b7022d7ddb16df1c&scope.system=all&role.id=159c6380cd984e998bb6a97ed7d98dac => generated 510 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:55:35.902815 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573d068-c419-4784-be20-8cfe03d7a5da None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:35.913325 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573d068-c419-4784-be20-8cfe03d7a5da None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:35.926099 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b28e59-3ddf-4451-af99-a8198b410822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:35.927049 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b28e59-3ddf-4451-af99-a8198b410822 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:35.927501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b28e59-3ddf-4451-af99-a8198b410822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:35.927940 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b28e59-3ddf-4451-af99-a8198b410822 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:35.939313 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b28e59-3ddf-4451-af99-a8198b410822 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:35.948890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-b6b28e59-3ddf-4451-af99-a8198b410822 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:55:35.948890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:35.948890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:35.948890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:35.948890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:35.948890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:35.948890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:35.948890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:35.948890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:35.948890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:35.948890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:35.948890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:35.948890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:35.948890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:35.948890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:35.948890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:35.949740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:35.949740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:35.949740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:35.949740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:35.949740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:35.949740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:35.949740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:35.949740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:35.949740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:35.949740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:35.949740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:35.949740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:35.949740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:35.949740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:35.949740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:35.949740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:35.950563 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:35.950563 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:35.950563 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:35.950563 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:35.950563 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:35.950563 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:35.950563 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:35.950563 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:35.950563 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:35.950563 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:35.950563 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:55:35.950563 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:35.952853 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 151/292] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Oct 22 23:55:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 50 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:35.959193 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96f9ab-7c61-48e8-887c-47ffc27369ba None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:35.970599 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96f9ab-7c61-48e8-887c-47ffc27369ba None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:35.980341 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c59e0-c5b8-4c21-8ef7-ab7fe1def2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:35.981161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c59e0-c5b8-4c21-8ef7-ab7fe1def2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:35.981908 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c59e0-c5b8-4c21-8ef7-ab7fe1def2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:35.982324 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c59e0-c5b8-4c21-8ef7-ab7fe1def2e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:35.983141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c59e0-c5b8-4c21-8ef7-ab7fe1def2e5 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:35.983582 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c59e0-c5b8-4c21-8ef7-ab7fe1def2e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:35.986075 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c59e0-c5b8-4c21-8ef7-ab7fe1def2e5 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:35.998854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 142/293] 10.222.0.13 () {56 vars in 1120 bytes} [Sat Oct 22 23:55:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:36.004439 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b28e59-3ddf-4451-af99-a8198b410822 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:36.015251 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b28e59-3ddf-4451-af99-a8198b410822 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:36.022325 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1aa21-49ea-4537-9874-bec686af69b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:36.022953 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1aa21-49ea-4537-9874-bec686af69b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:36.023233 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1aa21-49ea-4537-9874-bec686af69b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:36.023505 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1aa21-49ea-4537-9874-bec686af69b3 admin admin] PATH_INFO: `/v3/groups/c71f3dca5acb4cd8a4fef0be6705eef9` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:36.035474 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1aa21-49ea-4537-9874-bec686af69b3 admin admin] RBAC: Authorizing `identity:get_group(group_id=c71f3dca5acb4cd8a4fef0be6705eef9)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:36.037451 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1aa21-49ea-4537-9874-bec686af69b3 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:36.039067 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 152/294] 10.222.0.13 () {56 vars in 1201 bytes} [Sat Oct 22 23:55:36 2022] GET /identity/v3/groups/c71f3dca5acb4cd8a4fef0be6705eef9 => generated 229 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:36.044921 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c59e0-c5b8-4c21-8ef7-ab7fe1def2e5 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:36.060508 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c59e0-c5b8-4c21-8ef7-ab7fe1def2e5 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:36.072747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317929b-df4e-42c4-bb70-0fc3e4e5d49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:36.073621 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317929b-df4e-42c4-bb70-0fc3e4e5d49d admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:36.074073 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317929b-df4e-42c4-bb70-0fc3e4e5d49d admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:36.074516 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317929b-df4e-42c4-bb70-0fc3e4e5d49d admin admin] PATH_INFO: `/v3/projects/e646fb7093394a37b996a9f2520c5910` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:36.076191 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317929b-df4e-42c4-bb70-0fc3e4e5d49d admin admin] RBAC: Authorizing `identity:get_project(project_id=e646fb7093394a37b996a9f2520c5910)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:36.078373 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317929b-df4e-42c4-bb70-0fc3e4e5d49d admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:36.080801 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 143/295] 10.222.0.13 () {56 vars in 1207 bytes} [Sat Oct 22 23:55:36 2022] GET /identity/v3/projects/e646fb7093394a37b996a9f2520c5910 => generated 344 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:36.086205 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1aa21-49ea-4537-9874-bec686af69b3 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:36.113318 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1aa21-49ea-4537-9874-bec686af69b3 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:36.120829 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d4b9df-c7b6-4d03-8ceb-534ac74e05a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:36.121510 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4b9df-c7b6-4d03-8ceb-534ac74e05a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:36.121790 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4b9df-c7b6-4d03-8ceb-534ac74e05a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:36.122061 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d4b9df-c7b6-4d03-8ceb-534ac74e05a6 admin admin] PATH_INFO: `/v3/projects/e646fb7093394a37b996a9f2520c5910/groups/c71f3dca5acb4cd8a4fef0be6705eef9/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:36.129829 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4b9df-c7b6-4d03-8ceb-534ac74e05a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e646fb7093394a37b996a9f2520c5910, group_id=c71f3dca5acb4cd8a4fef0be6705eef9, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:36.132478 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d4b9df-c7b6-4d03-8ceb-534ac74e05a6 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:36.163782 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 153/296] 10.222.0.13 () {58 vars in 1463 bytes} [Sat Oct 22 23:55:36 2022] PUT /identity/v3/projects/e646fb7093394a37b996a9f2520c5910/groups/c71f3dca5acb4cd8a4fef0be6705eef9/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:55:36.850770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e6b55-3918-4f5d-b393-e4021546ca2e None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:36.851424 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e6b55-3918-4f5d-b393-e4021546ca2e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:36.851910 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e6b55-3918-4f5d-b393-e4021546ca2e None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:36.853491 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 144/297] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:36.859040 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033f4b4-3a78-4675-9e79-7da67d2eb574 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:36.859622 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033f4b4-3a78-4675-9e79-7da67d2eb574 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:36.860304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033f4b4-3a78-4675-9e79-7da67d2eb574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:36.879777 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-b033f4b4-3a78-4675-9e79-7da67d2eb574 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:36.884543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b033f4b4-3a78-4675-9e79-7da67d2eb574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:36.924629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f38038-ea3d-41ed-baa0-e638c119d550 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:36.925282 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f38038-ea3d-41ed-baa0-e638c119d550 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:36.925756 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f38038-ea3d-41ed-baa0-e638c119d550 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:36.927284 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 145/298] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:36.942063 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6e653-6245-4a41-b267-2c6aa3089b4c None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:36.942351 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6e653-6245-4a41-b267-2c6aa3089b4c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:36.942623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6e653-6245-4a41-b267-2c6aa3089b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:36.970372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-fae6e653-6245-4a41-b267-2c6aa3089b4c None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:36.971686 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-fae6e653-6245-4a41-b267-2c6aa3089b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:36.992322 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 154/299] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:37.067567 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 146/300] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 22 23:55:37.602092 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033f4b4-3a78-4675-9e79-7da67d2eb574 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:37.606000 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6e653-6245-4a41-b267-2c6aa3089b4c None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:37.639045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:37.785827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033f4b4-3a78-4675-9e79-7da67d2eb574 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:37.800612 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6e653-6245-4a41-b267-2c6aa3089b4c None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:37.803111 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:37.830444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1aa64-7c26-4ed7-ad22-fbfc39ef7baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:37.831355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1aa64-7c26-4ed7-ad22-fbfc39ef7baf None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:37.831814 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1aa64-7c26-4ed7-ad22-fbfc39ef7baf None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:37.832238 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1aa64-7c26-4ed7-ad22-fbfc39ef7baf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:37.852337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1aa64-7c26-4ed7-ad22-fbfc39ef7baf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:37.857797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-a5e1aa64-7c26-4ed7-ad22-fbfc39ef7baf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 22 23:55:37.857797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:37.857797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:37.857797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:37.857797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:37.857797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:37.857797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:37.857797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:37.857797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:37.857797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:37.857797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:37.857797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:37.857797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:37.857797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:37.857797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:37.857797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:37.858794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:37.858794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:37.858794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:37.858794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:37.858794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:37.858794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:37.858794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:37.858794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:37.858794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:37.858794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:37.858794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:37.858794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:37.858794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:37.858794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:37.858794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:37.858794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:37.859573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:37.859573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:37.859573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:37.859573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:37.859573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:37.859573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:37.859573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:37.859573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:37.859573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:37.859573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:37.859573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 22 23:55:37.859573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:37.859573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 155/301] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Oct 22 23:55:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:37.860296 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb37f6-8254-475b-80e0-d467e3121d20 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:37.861672 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb37f6-8254-475b-80e0-d467e3121d20 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:37.862196 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb37f6-8254-475b-80e0-d467e3121d20 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:37.863891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 156/302] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:37.865011 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800e85c-ca3f-4dac-9ebf-afc86054c374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:37.865956 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800e85c-ca3f-4dac-9ebf-afc86054c374 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:37.866429 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800e85c-ca3f-4dac-9ebf-afc86054c374 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:37.866890 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800e85c-ca3f-4dac-9ebf-afc86054c374 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:37.870069 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb37f6-8254-475b-80e0-d467e3121d20 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:37.888792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800e85c-ca3f-4dac-9ebf-afc86054c374 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:37.895549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-2800e85c-ca3f-4dac-9ebf-afc86054c374 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:37.895549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:37.895549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:37.895549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:37.895549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:37.895549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:37.895549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:37.895549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:37.895549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:37.895549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:37.895549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:37.895549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:37.895549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:37.895549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:37.895549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:37.895549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:37.896411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:37.896411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:37.896411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:37.896411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:37.896411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:37.896411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:37.896411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:37.896411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:37.896411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:37.896411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:37.896411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:37.896411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:37.896411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:37.896411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:37.896411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:37.896411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:37.897236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:37.897236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:37.897236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:37.897236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:37.897236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:37.897236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:37.897236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:37.897236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:37.897236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:37.897236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:37.897236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:37.897236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:37.899711 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 147/303] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:37.901102 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001217a-e968-4224-aea5-d2979aa70fcf None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:37.901549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001217a-e968-4224-aea5-d2979aa70fcf None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:37.901997 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c001217a-e968-4224-aea5-d2979aa70fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:37.924622 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-c001217a-e968-4224-aea5-d2979aa70fcf None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:37.926071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c001217a-e968-4224-aea5-d2979aa70fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:37.975141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:37.997083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb37f6-8254-475b-80e0-d467e3121d20 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:38.006624 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 148/304] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:38.008094 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c001217a-e968-4224-aea5-d2979aa70fcf None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:38.045211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da815431-3738-4b37-ae3d-f251d0058f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:38.045826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da815431-3738-4b37-ae3d-f251d0058f3c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:38.046108 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da815431-3738-4b37-ae3d-f251d0058f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:38.046382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da815431-3738-4b37-ae3d-f251d0058f3c None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:38.047228 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da815431-3738-4b37-ae3d-f251d0058f3c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:38.048023 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da815431-3738-4b37-ae3d-f251d0058f3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:38.050061 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da815431-3738-4b37-ae3d-f251d0058f3c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:38.059108 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 157/305] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:55:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:38.060359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a050a-6d7a-44f9-9db9-e124bf94a768 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:38.060743 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a050a-6d7a-44f9-9db9-e124bf94a768 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:38.061080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a050a-6d7a-44f9-9db9-e124bf94a768 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:38.062548 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 158/306] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:38.068714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a050a-6d7a-44f9-9db9-e124bf94a768 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:38.120513 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:38.122693 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c001217a-e968-4224-aea5-d2979aa70fcf None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:38.160371 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71572cfb-4804-424d-a736-65317b548b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:38.161125 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71572cfb-4804-424d-a736-65317b548b94 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:38.161409 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71572cfb-4804-424d-a736-65317b548b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:38.161681 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71572cfb-4804-424d-a736-65317b548b94 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:38.162410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71572cfb-4804-424d-a736-65317b548b94 None admin] RBAC: Adding query filter params (name=member) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:38.162712 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71572cfb-4804-424d-a736-65317b548b94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:38.164257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71572cfb-4804-424d-a736-65317b548b94 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:38.175020 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 149/307] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Oct 22 23:55:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:38.176231 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556ba52-9af0-4d0c-ad46-c015097e5339 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:38.176534 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556ba52-9af0-4d0c-ad46-c015097e5339 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:38.176814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556ba52-9af0-4d0c-ad46-c015097e5339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:38.189161 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a050a-6d7a-44f9-9db9-e124bf94a768 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:38.198393 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-1556ba52-9af0-4d0c-ad46-c015097e5339 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:38.199578 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-1556ba52-9af0-4d0c-ad46-c015097e5339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:38.230272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd926c-fc43-4477-99c4-562408658f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:38.230888 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd926c-fc43-4477-99c4-562408658f46 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:38.231174 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd926c-fc43-4477-99c4-562408658f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:38.231455 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd926c-fc43-4477-99c4-562408658f46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:38.247570 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:38.275617 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 150/308] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:38.276892 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556ba52-9af0-4d0c-ad46-c015097e5339 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:38.306881 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd926c-fc43-4477-99c4-562408658f46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:38.308350 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbd926c-fc43-4477-99c4-562408658f46 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:38.378809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:38.393430 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556ba52-9af0-4d0c-ad46-c015097e5339 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:38.419348 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 159/309] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:55:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 22 23:55:38.420746 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd926c-fc43-4477-99c4-562408658f46 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:38.448281 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227a773-dfdb-4ed0-81fe-73d842372f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:38.449159 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227a773-dfdb-4ed0-81fe-73d842372f55 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:38.449763 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227a773-dfdb-4ed0-81fe-73d842372f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:38.450098 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227a773-dfdb-4ed0-81fe-73d842372f55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:38.471550 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:38.526263 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd926c-fc43-4477-99c4-562408658f46 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:38.540175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1227a773-dfdb-4ed0-81fe-73d842372f55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:38.541893 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1227a773-dfdb-4ed0-81fe-73d842372f55 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:38.565169 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e6b04-d2fe-4063-aef8-9f0a57fd82f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:38.565962 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e6b04-d2fe-4063-aef8-9f0a57fd82f2 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:38.566613 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e6b04-d2fe-4063-aef8-9f0a57fd82f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:38.566989 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e6b04-d2fe-4063-aef8-9f0a57fd82f2 None admin] PATH_INFO: `/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:38.568281 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e6b04-d2fe-4063-aef8-9f0a57fd82f2 None admin] RBAC: Authorizing `identity:get_project(project_id=4d6b66c470f74a4dac6f66e6fcee2a51)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:38.571281 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e6b04-d2fe-4063-aef8-9f0a57fd82f2 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:38.577670 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 160/310] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:55:38 2022] GET /identity/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:55:38.578608 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e6b04-d2fe-4063-aef8-9f0a57fd82f2 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:38.645778 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:38.665099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 151/311] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:55:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 22 23:55:38.669443 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227a773-dfdb-4ed0-81fe-73d842372f55 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:38.723000 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e6b04-d2fe-4063-aef8-9f0a57fd82f2 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:38.738485 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:38.773102 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d54aa-b887-4bf6-a1c0-7e02ca906e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:38.773804 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d54aa-b887-4bf6-a1c0-7e02ca906e90 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:38.774260 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d54aa-b887-4bf6-a1c0-7e02ca906e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:38.774536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d54aa-b887-4bf6-a1c0-7e02ca906e90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:38.787613 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d54aa-b887-4bf6-a1c0-7e02ca906e90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:38.794705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-728d54aa-b887-4bf6-a1c0-7e02ca906e90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:38.794705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:38.794705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:38.794705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:38.794705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:38.794705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:38.794705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:38.794705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:38.794705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:38.794705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:38.794705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:38.794705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:38.794705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:38.794705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:38.794705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:38.794705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:38.795635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:38.795635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:38.795635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:38.795635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:38.795635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:38.795635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:38.795635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:38.795635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:38.795635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:38.795635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:38.795635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:38.795635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:38.795635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:38.795635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:38.795635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:38.795635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:38.796826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:38.796826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:38.796826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:38.796826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:38.796826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:38.796826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:38.796826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:38.796826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:38.796826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:38.796826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:38.796826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 22 23:55:38.796826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:38.796826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 161/312] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:38.796826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d54aa-b887-4bf6-a1c0-7e02ca906e90 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:38.799609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227a773-dfdb-4ed0-81fe-73d842372f55 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:38.848971 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28b2c3c-d01d-4ab6-9175-8b98be15315b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:38.849641 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b2c3c-d01d-4ab6-9175-8b98be15315b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:38.850364 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b2c3c-d01d-4ab6-9175-8b98be15315b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:38.850645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28b2c3c-d01d-4ab6-9175-8b98be15315b None admin] PATH_INFO: `/v3/groups/8a1180354f9445f49e7ad23bac3f4557` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:38.852091 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28b2c3c-d01d-4ab6-9175-8b98be15315b None admin] RBAC: Authorizing `identity:get_group(group_id=8a1180354f9445f49e7ad23bac3f4557)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:38.853594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28b2c3c-d01d-4ab6-9175-8b98be15315b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:38.855501 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 152/313] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Oct 22 23:55:38 2022] GET /identity/v3/groups/8a1180354f9445f49e7ad23bac3f4557 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:38.856289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28b2c3c-d01d-4ab6-9175-8b98be15315b None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:38.922311 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:38.924152 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d54aa-b887-4bf6-a1c0-7e02ca906e90 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:38.979616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc83e34-b935-4060-b902-f443f84aa746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:38.980272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc83e34-b935-4060-b902-f443f84aa746 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:38.980602 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc83e34-b935-4060-b902-f443f84aa746 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:38.981217 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc83e34-b935-4060-b902-f443f84aa746 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:38.984361 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28b2c3c-d01d-4ab6-9175-8b98be15315b None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:38.995987 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc83e34-b935-4060-b902-f443f84aa746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:39.002502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-9cc83e34-b935-4060-b902-f443f84aa746 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 22 23:55:39.002502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:39.002502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:39.002502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:39.002502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:39.002502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:39.002502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:39.002502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:39.002502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:39.002502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:39.002502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:39.002502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:39.002502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:39.002502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:39.002502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:39.002502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:39.003926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:39.003926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:39.003926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:39.003926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:39.003926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:39.003926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:39.003926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:39.003926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:39.003926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:39.003926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:39.003926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:39.003926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:39.003926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:39.003926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:39.003926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:39.003926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:39.004747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:39.004747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:39.004747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:39.004747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:39.004747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:39.004747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:39.004747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:39.004747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:39.004747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:39.004747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:39.004747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 22 23:55:39.004747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:39.005926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 162/314] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Oct 22 23:55:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:39.008580 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc83e34-b935-4060-b902-f443f84aa746 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:39.036253 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a72d1-cee0-48d3-86a6-fcc888038020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:39.036790 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a72d1-cee0-48d3-86a6-fcc888038020 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:39.036790 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a72d1-cee0-48d3-86a6-fcc888038020 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:39.036958 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432a72d1-cee0-48d3-86a6-fcc888038020 None admin] PATH_INFO: `/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:39.039019 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432a72d1-cee0-48d3-86a6-fcc888038020 None admin] RBAC: Authorizing `identity:get_project(project_id=4d6b66c470f74a4dac6f66e6fcee2a51)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:39.040480 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432a72d1-cee0-48d3-86a6-fcc888038020 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:39.042647 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 153/315] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:55:38 2022] GET /identity/v3/projects/4d6b66c470f74a4dac6f66e6fcee2a51 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:39.043354 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a72d1-cee0-48d3-86a6-fcc888038020 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:39.050012 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:39.136379 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc83e34-b935-4060-b902-f443f84aa746 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:39.159445 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432a72d1-cee0-48d3-86a6-fcc888038020 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:39.175905 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121c89c-2238-410b-8d27-7f7a419e79d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:39.176445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121c89c-2238-410b-8d27-7f7a419e79d5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:39.176758 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121c89c-2238-410b-8d27-7f7a419e79d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:39.177033 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121c89c-2238-410b-8d27-7f7a419e79d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:39.177694 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121c89c-2238-410b-8d27-7f7a419e79d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:39.177994 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121c89c-2238-410b-8d27-7f7a419e79d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:39.179362 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121c89c-2238-410b-8d27-7f7a419e79d5 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:39.184298 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:39.191087 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 163/316] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Oct 22 23:55:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:39.192521 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121c89c-2238-410b-8d27-7f7a419e79d5 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:39.200229 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db49a75-0e5e-46e2-957c-8e445ed34000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:39.200863 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db49a75-0e5e-46e2-957c-8e445ed34000 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:39.202175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db49a75-0e5e-46e2-957c-8e445ed34000 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:39.202462 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db49a75-0e5e-46e2-957c-8e445ed34000 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:39.203201 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db49a75-0e5e-46e2-957c-8e445ed34000 None admin] RBAC: Adding query filter params (group.id=8a1180354f9445f49e7ad23bac3f4557, role.id=cf8116e4e82748ac872d8763d113ba61, scope.project.id=4d6b66c470f74a4dac6f66e6fcee2a51) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:39.203510 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db49a75-0e5e-46e2-957c-8e445ed34000 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:39.205114 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db49a75-0e5e-46e2-957c-8e445ed34000 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:39.211840 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 154/317] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Oct 22 23:55:39 2022] GET /identity/v3/role_assignments?group.id=8a1180354f9445f49e7ad23bac3f4557&scope.project.id=4d6b66c470f74a4dac6f66e6fcee2a51&role.id=cf8116e4e82748ac872d8763d113ba61 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:39.214933 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db49a75-0e5e-46e2-957c-8e445ed34000 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:39.324118 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:39.338327 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121c89c-2238-410b-8d27-7f7a419e79d5 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:39.362852 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db49a75-0e5e-46e2-957c-8e445ed34000 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:39.387610 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730f84b-03d5-4ec3-a1a0-24461961b596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:39.389016 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730f84b-03d5-4ec3-a1a0-24461961b596 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:39.389016 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730f84b-03d5-4ec3-a1a0-24461961b596 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:39.389016 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730f84b-03d5-4ec3-a1a0-24461961b596 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:39.389446 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730f84b-03d5-4ec3-a1a0-24461961b596 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:39.389748 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730f84b-03d5-4ec3-a1a0-24461961b596 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:39.391940 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730f84b-03d5-4ec3-a1a0-24461961b596 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:39.402382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 164/318] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:55:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:39.404588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdceef17-ed36-48d6-9a85-85ad04bd2d9b None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:39.404884 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdceef17-ed36-48d6-9a85-85ad04bd2d9b None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:39.405158 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdceef17-ed36-48d6-9a85-85ad04bd2d9b None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:39.406728 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 165/319] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:39.409828 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdceef17-ed36-48d6-9a85-85ad04bd2d9b None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:39.417700 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074296f-3b17-4db6-b188-73435c5ca8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:39.418347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074296f-3b17-4db6-b188-73435c5ca8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:39.419373 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074296f-3b17-4db6-b188-73435c5ca8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:39.419373 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074296f-3b17-4db6-b188-73435c5ca8b0 None admin] PATH_INFO: `/v3/groups/8a1180354f9445f49e7ad23bac3f4557` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:39.420718 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074296f-3b17-4db6-b188-73435c5ca8b0 None admin] RBAC: Authorizing `identity:get_group(group_id=8a1180354f9445f49e7ad23bac3f4557)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:39.422174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074296f-3b17-4db6-b188-73435c5ca8b0 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:39.423895 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 155/320] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Oct 22 23:55:39 2022] GET /identity/v3/groups/8a1180354f9445f49e7ad23bac3f4557 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:39.426441 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074296f-3b17-4db6-b188-73435c5ca8b0 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:39.498108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:39.535235 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdceef17-ed36-48d6-9a85-85ad04bd2d9b None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:39.553479 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074296f-3b17-4db6-b188-73435c5ca8b0 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:39.574094 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3185ac-cf7e-42f2-b88b-cdaee414efbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:39.574719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3185ac-cf7e-42f2-b88b-cdaee414efbc None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:39.575002 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3185ac-cf7e-42f2-b88b-cdaee414efbc None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:39.575282 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3185ac-cf7e-42f2-b88b-cdaee414efbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:39.595727 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2af7e-e835-4dbe-88b2-2249d96880f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:39.596342 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2af7e-e835-4dbe-88b2-2249d96880f1 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:39.596702 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2af7e-e835-4dbe-88b2-2249d96880f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:39.596983 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2af7e-e835-4dbe-88b2-2249d96880f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:39.606720 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:39.657850 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3185ac-cf7e-42f2-b88b-cdaee414efbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:39.659314 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3185ac-cf7e-42f2-b88b-cdaee414efbc None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:39.675901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2af7e-e835-4dbe-88b2-2249d96880f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:39.677430 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2af7e-e835-4dbe-88b2-2249d96880f1 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:39.730797 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:39.758775 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 166/321] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:55:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 22 23:55:39.760778 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b866f6f8-c707-4667-8296-429f546bcb0d None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:39.761069 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b866f6f8-c707-4667-8296-429f546bcb0d None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:39.761347 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b866f6f8-c707-4667-8296-429f546bcb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:39.777885 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 156/322] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:55:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 22 23:55:39.778938 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-b866f6f8-c707-4667-8296-429f546bcb0d None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:39.779971 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2af7e-e835-4dbe-88b2-2249d96880f1 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:39.780166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b866f6f8-c707-4667-8296-429f546bcb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:39.830916 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:39.848803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 167/323] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:39.849715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b866f6f8-c707-4667-8296-429f546bcb0d None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:39.884959 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2af7e-e835-4dbe-88b2-2249d96880f1 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:39.921057 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c8a87-f5eb-4cb5-9f6d-aa5bbd567823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:39.922201 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c8a87-f5eb-4cb5-9f6d-aa5bbd567823 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:39.922861 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c8a87-f5eb-4cb5-9f6d-aa5bbd567823 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:39.923354 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c8a87-f5eb-4cb5-9f6d-aa5bbd567823 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:39.924271 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c8a87-f5eb-4cb5-9f6d-aa5bbd567823 None admin] RBAC: Adding query filter params (group.id=8a1180354f9445f49e7ad23bac3f4557, role.id=4bd2874c95634a50aac9fccbddc7c674, scope.project.id=4d6b66c470f74a4dac6f66e6fcee2a51) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:39.924867 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c8a87-f5eb-4cb5-9f6d-aa5bbd567823 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:39.926627 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c8a87-f5eb-4cb5-9f6d-aa5bbd567823 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:39.933175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 157/324] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Oct 22 23:55:39 2022] GET /identity/v3/role_assignments?group.id=8a1180354f9445f49e7ad23bac3f4557&scope.project.id=4d6b66c470f74a4dac6f66e6fcee2a51&role.id=4bd2874c95634a50aac9fccbddc7c674 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:39.934265 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c8a87-f5eb-4cb5-9f6d-aa5bbd567823 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:39.968951 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:39.988114 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b866f6f8-c707-4667-8296-429f546bcb0d None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:40.029080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe9b8a-e0b5-48e1-85a4-cdc9e07f97af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:40.030465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe9b8a-e0b5-48e1-85a4-cdc9e07f97af None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:40.030755 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe9b8a-e0b5-48e1-85a4-cdc9e07f97af None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:40.031041 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe9b8a-e0b5-48e1-85a4-cdc9e07f97af None admin] PATH_INFO: `/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:40.032427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe9b8a-e0b5-48e1-85a4-cdc9e07f97af None admin] RBAC: Authorizing `identity:get_project(project_id=79baed4da64f4fc2b256aa6fc529fbd2)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:40.033991 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe9b8a-e0b5-48e1-85a4-cdc9e07f97af None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:40.036134 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 168/325] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:55:39 2022] GET /identity/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:40.045584 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe9b8a-e0b5-48e1-85a4-cdc9e07f97af None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:40.078073 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c8a87-f5eb-4cb5-9f6d-aa5bbd567823 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:40.121977 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:40.122410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195db7f-f3bb-41ab-8554-28a0f437e065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:40.123121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195db7f-f3bb-41ab-8554-28a0f437e065 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:40.123414 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195db7f-f3bb-41ab-8554-28a0f437e065 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:40.123693 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195db7f-f3bb-41ab-8554-28a0f437e065 None admin] PATH_INFO: `/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:40.124948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4195db7f-f3bb-41ab-8554-28a0f437e065 None admin] RBAC: Authorizing `identity:get_project(project_id=79baed4da64f4fc2b256aa6fc529fbd2)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:40.127365 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4195db7f-f3bb-41ab-8554-28a0f437e065 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:40.129085 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 158/326] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:55:39 2022] GET /identity/v3/projects/79baed4da64f4fc2b256aa6fc529fbd2 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:40.129952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195db7f-f3bb-41ab-8554-28a0f437e065 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:40.177974 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe9b8a-e0b5-48e1-85a4-cdc9e07f97af None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:40.215105 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f8df9-8c1c-4453-a324-198da1af12d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:40.215664 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f8df9-8c1c-4453-a324-198da1af12d6 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:40.215945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f8df9-8c1c-4453-a324-198da1af12d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:40.216221 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f8df9-8c1c-4453-a324-198da1af12d6 None admin] PATH_INFO: `/v3/groups/8a1180354f9445f49e7ad23bac3f4557` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:40.217355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f8df9-8c1c-4453-a324-198da1af12d6 None admin] RBAC: Authorizing `identity:get_group(group_id=8a1180354f9445f49e7ad23bac3f4557)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:40.218765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f8df9-8c1c-4453-a324-198da1af12d6 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:40.221034 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 169/327] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Oct 22 23:55:40 2022] GET /identity/v3/groups/8a1180354f9445f49e7ad23bac3f4557 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:40.221814 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f8df9-8c1c-4453-a324-198da1af12d6 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:40.237332 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:40.254103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195db7f-f3bb-41ab-8554-28a0f437e065 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:40.296681 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31509167-b950-4833-a0c2-12a3c1846ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:40.298665 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31509167-b950-4833-a0c2-12a3c1846ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:40.298917 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31509167-b950-4833-a0c2-12a3c1846ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:40.299014 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31509167-b950-4833-a0c2-12a3c1846ab8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:40.313120 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31509167-b950-4833-a0c2-12a3c1846ab8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:40.319192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-31509167-b950-4833-a0c2-12a3c1846ab8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:55:40.319192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:40.319192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:40.319192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:40.319192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:40.319192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:40.319192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:40.319192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:40.319192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:40.319192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:40.319192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:40.319192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:40.319192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:40.319192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:40.319192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:40.319192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:40.320122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:40.320122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:40.320122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:40.320122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:40.320122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:40.320122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:40.320122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:40.320122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:40.320122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:40.320122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:40.320122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:40.320122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:40.320122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:40.320122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:40.320122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:40.320122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:40.320984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:40.320984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:40.320984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:40.320984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:40.320984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:40.320984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:40.320984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:40.320984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:40.320984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:40.320984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:40.320984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:55:40.320984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:40.320984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 159/328] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Oct 22 23:55:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:40.321646 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31509167-b950-4833-a0c2-12a3c1846ab8 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:40.340544 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f8df9-8c1c-4453-a324-198da1af12d6 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:40.371098 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:40.382406 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c373443-8892-47fd-a1d6-395a4a4ca8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:40.383043 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c373443-8892-47fd-a1d6-395a4a4ca8cb None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:40.383323 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c373443-8892-47fd-a1d6-395a4a4ca8cb None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:40.383595 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c373443-8892-47fd-a1d6-395a4a4ca8cb None admin] PATH_INFO: `/v3/groups/8a1180354f9445f49e7ad23bac3f4557` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:40.384726 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c373443-8892-47fd-a1d6-395a4a4ca8cb None admin] RBAC: Authorizing `identity:get_group(group_id=8a1180354f9445f49e7ad23bac3f4557)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:40.386142 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c373443-8892-47fd-a1d6-395a4a4ca8cb None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:40.387768 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 170/329] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Oct 22 23:55:40 2022] GET /identity/v3/groups/8a1180354f9445f49e7ad23bac3f4557 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:40.388562 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c373443-8892-47fd-a1d6-395a4a4ca8cb None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:40.445387 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31509167-b950-4833-a0c2-12a3c1846ab8 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:40.486216 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d76af-627e-413c-b6ab-25543fbfb021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:40.486771 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d76af-627e-413c-b6ab-25543fbfb021 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:40.487050 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d76af-627e-413c-b6ab-25543fbfb021 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:40.487323 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d76af-627e-413c-b6ab-25543fbfb021 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:40.488053 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d76af-627e-413c-b6ab-25543fbfb021 None admin] RBAC: Adding query filter params (group.id=8a1180354f9445f49e7ad23bac3f4557, role.id=4bd2874c95634a50aac9fccbddc7c674, scope.project.id=79baed4da64f4fc2b256aa6fc529fbd2) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:40.488358 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d76af-627e-413c-b6ab-25543fbfb021 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:40.489842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d76af-627e-413c-b6ab-25543fbfb021 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:40.495956 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 160/330] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Oct 22 23:55:40 2022] GET /identity/v3/role_assignments?group.id=8a1180354f9445f49e7ad23bac3f4557&scope.project.id=79baed4da64f4fc2b256aa6fc529fbd2&role.id=4bd2874c95634a50aac9fccbddc7c674 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:40.496596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d76af-627e-413c-b6ab-25543fbfb021 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:40.510126 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c373443-8892-47fd-a1d6-395a4a4ca8cb None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:40.512828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:40.548686 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968deb16-600e-484b-8d7b-ecd8536c1086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:40.549321 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968deb16-600e-484b-8d7b-ecd8536c1086 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:40.549642 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968deb16-600e-484b-8d7b-ecd8536c1086 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:40.549880 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968deb16-600e-484b-8d7b-ecd8536c1086 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:40.550664 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968deb16-600e-484b-8d7b-ecd8536c1086 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:40.550973 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968deb16-600e-484b-8d7b-ecd8536c1086 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:40.552374 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968deb16-600e-484b-8d7b-ecd8536c1086 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:40.568312 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 171/331] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:40.569249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968deb16-600e-484b-8d7b-ecd8536c1086 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:40.614068 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d76af-627e-413c-b6ab-25543fbfb021 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:40.629632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:40.652511 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f674c26-7b77-49c0-8510-b3704c3182bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:40.653134 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f674c26-7b77-49c0-8510-b3704c3182bf None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:40.653417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f674c26-7b77-49c0-8510-b3704c3182bf None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:40.653694 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f674c26-7b77-49c0-8510-b3704c3182bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:40.654436 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f674c26-7b77-49c0-8510-b3704c3182bf None admin] RBAC: Adding query filter params (group.id=8a1180354f9445f49e7ad23bac3f4557, role.id=cf8116e4e82748ac872d8763d113ba61, scope.project.id=79baed4da64f4fc2b256aa6fc529fbd2) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:40.654745 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f674c26-7b77-49c0-8510-b3704c3182bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:40.656219 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f674c26-7b77-49c0-8510-b3704c3182bf None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:40.664659 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 161/332] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Oct 22 23:55:40 2022] GET /identity/v3/role_assignments?group.id=8a1180354f9445f49e7ad23bac3f4557&scope.project.id=79baed4da64f4fc2b256aa6fc529fbd2&role.id=cf8116e4e82748ac872d8763d113ba61 => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:40.697013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968deb16-600e-484b-8d7b-ecd8536c1086 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:40.739911 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1cfc46-b04f-43ef-a13f-29c49936cf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:40.740458 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cfc46-b04f-43ef-a13f-29c49936cf10 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:40.740771 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cfc46-b04f-43ef-a13f-29c49936cf10 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:40.741050 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1cfc46-b04f-43ef-a13f-29c49936cf10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:40.820783 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1cfc46-b04f-43ef-a13f-29c49936cf10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:40.822317 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1cfc46-b04f-43ef-a13f-29c49936cf10 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:40.879393 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:40.930958 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 172/333] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:55:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 22 23:55:40.936715 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f674c26-7b77-49c0-8510-b3704c3182bf None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:41.012030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:41.038117 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f674c26-7b77-49c0-8510-b3704c3182bf None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:41.073309 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6c37b-b064-46bf-a403-cfd5db09dcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:41.073963 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6c37b-b064-46bf-a403-cfd5db09dcbf None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:41.074244 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6c37b-b064-46bf-a403-cfd5db09dcbf None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:41.074516 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6c37b-b064-46bf-a403-cfd5db09dcbf None admin] PATH_INFO: `/v3/projects/e646fb7093394a37b996a9f2520c5910` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:41.075615 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6c37b-b064-46bf-a403-cfd5db09dcbf None admin] RBAC: Authorizing `identity:get_project(project_id=e646fb7093394a37b996a9f2520c5910)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:41.077066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6c37b-b064-46bf-a403-cfd5db09dcbf None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:41.078663 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 162/334] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:55:40 2022] GET /identity/v3/projects/e646fb7093394a37b996a9f2520c5910 => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:41.084755 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1cfc46-b04f-43ef-a13f-29c49936cf10 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:41.182285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1cfc46-b04f-43ef-a13f-29c49936cf10 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:41.215354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1ae8b-baf6-49e1-9d35-b5a445ce85ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:41.215968 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1ae8b-baf6-49e1-9d35-b5a445ce85ae None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:41.216247 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1ae8b-baf6-49e1-9d35-b5a445ce85ae None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:41.216542 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1ae8b-baf6-49e1-9d35-b5a445ce85ae None admin] PATH_INFO: `/v3/groups/c71f3dca5acb4cd8a4fef0be6705eef9` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:41.217610 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a1ae8b-baf6-49e1-9d35-b5a445ce85ae None admin] RBAC: Authorizing `identity:get_group(group_id=c71f3dca5acb4cd8a4fef0be6705eef9)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:41.219009 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a1ae8b-baf6-49e1-9d35-b5a445ce85ae None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:41.221596 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 173/335] 10.222.0.13 () {56 vars in 1180 bytes} [Sat Oct 22 23:55:41 2022] GET /identity/v3/groups/c71f3dca5acb4cd8a4fef0be6705eef9 => generated 229 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:41.227350 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6c37b-b064-46bf-a403-cfd5db09dcbf None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:41.322853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6c37b-b064-46bf-a403-cfd5db09dcbf None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:41.354685 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a94f7f9-f5b2-42bb-b18b-25ae170abc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:41.355269 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94f7f9-f5b2-42bb-b18b-25ae170abc0b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:41.355557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94f7f9-f5b2-42bb-b18b-25ae170abc0b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:41.355833 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a94f7f9-f5b2-42bb-b18b-25ae170abc0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:41.356576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a94f7f9-f5b2-42bb-b18b-25ae170abc0b None admin] RBAC: Adding query filter params (group.id=c71f3dca5acb4cd8a4fef0be6705eef9, role.id=839a55263e2c4cbaa2f945b75cf298d6, scope.project.id=e646fb7093394a37b996a9f2520c5910) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:41.356889 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a94f7f9-f5b2-42bb-b18b-25ae170abc0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:41.358320 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a94f7f9-f5b2-42bb-b18b-25ae170abc0b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:41.364023 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 163/336] 10.222.0.13 () {56 vars in 1376 bytes} [Sat Oct 22 23:55:41 2022] GET /identity/v3/role_assignments?group.id=c71f3dca5acb4cd8a4fef0be6705eef9&scope.project.id=e646fb7093394a37b996a9f2520c5910&role.id=839a55263e2c4cbaa2f945b75cf298d6 => generated 611 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:43.886208 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ae3f9-ab2f-4530-b6d5-f079e2b2bc93 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:43.887226 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ae3f9-ab2f-4530-b6d5-f079e2b2bc93 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:43.887713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ae3f9-ab2f-4530-b6d5-f079e2b2bc93 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:43.889585 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 174/337] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:43.904330 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ea287-b8f2-4436-bf64-3c476aa22ee4 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:43.905208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ea287-b8f2-4436-bf64-3c476aa22ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:43.905675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ea287-b8f2-4436-bf64-3c476aa22ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:43.923954 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:43.933604 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-f11ea287-b8f2-4436-bf64-3c476aa22ee4 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:43.935158 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f11ea287-b8f2-4436-bf64-3c476aa22ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:44.035077 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 164/338] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:44.068278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b20cdb-7960-4ca9-a833-f569df32dca1 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:44.068933 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b20cdb-7960-4ca9-a833-f569df32dca1 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:44.069466 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b20cdb-7960-4ca9-a833-f569df32dca1 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:44.071293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 175/339] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:44.076309 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36a9e4-7885-4203-81a4-fd6b9d7975e6 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:44.076644 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36a9e4-7885-4203-81a4-fd6b9d7975e6 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:44.076928 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36a9e4-7885-4203-81a4-fd6b9d7975e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:44.101915 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-cd36a9e4-7885-4203-81a4-fd6b9d7975e6 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:44.103118 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-cd36a9e4-7885-4203-81a4-fd6b9d7975e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:44.207276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 165/340] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:44.334745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b20cdb-7960-4ca9-a833-f569df32dca1 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:44.363225 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:44.367202 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80f187-f8c7-4244-b522-54880857e86b None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:44.368449 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80f187-f8c7-4244-b522-54880857e86b None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:44.369152 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80f187-f8c7-4244-b522-54880857e86b None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:44.372153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 166/341] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:44.377346 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199b1bd-202d-4de3-87b0-a38f1d42f81f None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:44.377987 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199b1bd-202d-4de3-87b0-a38f1d42f81f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:44.378469 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199b1bd-202d-4de3-87b0-a38f1d42f81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:44.397415 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-5199b1bd-202d-4de3-87b0-a38f1d42f81f None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:44.399122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-5199b1bd-202d-4de3-87b0-a38f1d42f81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:44.467441 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:44.485480 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 167/342] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 22 23:55:44.525165 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b20cdb-7960-4ca9-a833-f569df32dca1 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:44.579927 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9905b4-c14f-4a8b-bf60-dbd79a1d0fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:44.580585 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9905b4-c14f-4a8b-bf60-dbd79a1d0fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:44.580868 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9905b4-c14f-4a8b-bf60-dbd79a1d0fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:44.581138 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9905b4-c14f-4a8b-bf60-dbd79a1d0fe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:44.583679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199b1bd-202d-4de3-87b0-a38f1d42f81f None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:44.594062 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9905b4-c14f-4a8b-bf60-dbd79a1d0fe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:44.596920 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9905b4-c14f-4a8b-bf60-dbd79a1d0fe6 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:44.615837 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-8f9905b4-c14f-4a8b-bf60-dbd79a1d0fe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:44.615837 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:44.615837 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:44.615837 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:44.615837 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:44.615837 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:44.615837 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:44.615837 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:44.615837 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:44.615837 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:44.615837 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:44.615837 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:44.615837 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:55:44.615837 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:55:44.615837 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:55:44.615837 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:55:44.616899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:44.616899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:44.616899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:44.616899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:44.616899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:44.616899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:44.616899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:44.616899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:44.616899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:44.616899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:44.616899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:44.616899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:44.616899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:44.616899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:44.616899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:44.616899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:44.616899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:55:44.618392 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:55:44.618392 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:44.618392 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:44.618392 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 176/343] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:44.625423 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9905b4-c14f-4a8b-bf60-dbd79a1d0fe6 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:44.706697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199b1bd-202d-4de3-87b0-a38f1d42f81f None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:44.756984 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:44.785197 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83992b7-852b-4ab9-aa4e-197f17fefafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:44.785839 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83992b7-852b-4ab9-aa4e-197f17fefafa None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:44.786118 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83992b7-852b-4ab9-aa4e-197f17fefafa None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:44.786391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83992b7-852b-4ab9-aa4e-197f17fefafa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:44.789084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9905b4-c14f-4a8b-bf60-dbd79a1d0fe6 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:44.805459 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83992b7-852b-4ab9-aa4e-197f17fefafa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:44.807109 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83992b7-852b-4ab9-aa4e-197f17fefafa None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:44.823605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-a83992b7-852b-4ab9-aa4e-197f17fefafa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:44.823605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:44.823605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:44.823605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:44.823605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:44.823605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:44.823605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:44.823605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:44.823605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:44.823605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:44.823605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:44.823605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:44.823605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:55:44.823605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:55:44.823605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:55:44.823605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:55:44.824725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:44.824725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:44.824725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:44.824725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:44.824725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:44.824725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:44.824725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:44.824725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:44.824725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:44.824725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:44.824725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:44.824725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:44.824725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:44.824725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:44.824725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:44.824725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:44.824725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:55:44.826184 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:55:44.826184 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:44.826184 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:44.826184 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 168/344] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:55:44.826467 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40ce81-89a2-4e06-8e1f-390999aed34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:44.826576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83992b7-852b-4ab9-aa4e-197f17fefafa None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:44.826681 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40ce81-89a2-4e06-8e1f-390999aed34f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:44.828667 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40ce81-89a2-4e06-8e1f-390999aed34f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:44.828667 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40ce81-89a2-4e06-8e1f-390999aed34f None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:44.828667 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef40ce81-89a2-4e06-8e1f-390999aed34f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:44.828667 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef40ce81-89a2-4e06-8e1f-390999aed34f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:44.830484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef40ce81-89a2-4e06-8e1f-390999aed34f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:44.843241 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 177/345] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:44.844168 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40ce81-89a2-4e06-8e1f-390999aed34f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:44.881157 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:44.948211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83992b7-852b-4ab9-aa4e-197f17fefafa None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:44.959779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40ce81-89a2-4e06-8e1f-390999aed34f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:44.995660 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0aac3-24f6-4e54-aeda-64bede4622f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:44.996687 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0aac3-24f6-4e54-aeda-64bede4622f1 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:44.996975 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0aac3-24f6-4e54-aeda-64bede4622f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:44.997256 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0aac3-24f6-4e54-aeda-64bede4622f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:45.000565 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:45.008574 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfaa679-d259-40f7-ae4b-f2528ceb7bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:45.009732 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfaa679-d259-40f7-ae4b-f2528ceb7bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:45.009850 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0aac3-24f6-4e54-aeda-64bede4622f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:45.010346 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfaa679-d259-40f7-ae4b-f2528ceb7bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:45.010838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfaa679-d259-40f7-ae4b-f2528ceb7bf2 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:45.011306 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0aac3-24f6-4e54-aeda-64bede4622f1 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:45.011916 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfaa679-d259-40f7-ae4b-f2528ceb7bf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:45.012420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfaa679-d259-40f7-ae4b-f2528ceb7bf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:45.014730 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfaa679-d259-40f7-ae4b-f2528ceb7bf2 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:45.028458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-5ae0aac3-24f6-4e54-aeda-64bede4622f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:45.028458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:45.028458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:45.028458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:45.028458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:45.028458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:45.028458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:45.028458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:45.028458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:45.028458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:45.028458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:45.028458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:45.028458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:55:45.028458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:55:45.028458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:55:45.028458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:55:45.029427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:45.029427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:45.029427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:45.029427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:45.029427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:45.029427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:45.029427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:45.029427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:45.029427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:45.029427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:45.029427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:45.029427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:45.029427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:45.029427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:45.029427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:45.029427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:45.029427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:55:45.030271 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:55:45.030271 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:45.030271 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:45.030271 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 169/346] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:45.030906 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0aac3-24f6-4e54-aeda-64bede4622f1 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:45.031372 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 178/347] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:45.041205 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfaa679-d259-40f7-ae4b-f2528ceb7bf2 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:45.140706 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:45.144156 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0aac3-24f6-4e54-aeda-64bede4622f1 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:45.153366 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfaa679-d259-40f7-ae4b-f2528ceb7bf2 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:45.178150 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1b96a-5a5e-4230-bdf7-a26f765ba7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:45.178853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1b96a-5a5e-4230-bdf7-a26f765ba7b3 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:45.179137 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1b96a-5a5e-4230-bdf7-a26f765ba7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:45.179415 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1b96a-5a5e-4230-bdf7-a26f765ba7b3 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:45.180057 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1b96a-5a5e-4230-bdf7-a26f765ba7b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:45.181502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1b96a-5a5e-4230-bdf7-a26f765ba7b3 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:45.190374 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab6dc3-f05a-4aec-b8b0-ad78fced5257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:45.191317 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab6dc3-f05a-4aec-b8b0-ad78fced5257 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:45.192174 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab6dc3-f05a-4aec-b8b0-ad78fced5257 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:45.192708 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab6dc3-f05a-4aec-b8b0-ad78fced5257 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:45.193669 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ab6dc3-f05a-4aec-b8b0-ad78fced5257 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:45.194099 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ab6dc3-f05a-4aec-b8b0-ad78fced5257 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:45.196203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ab6dc3-f05a-4aec-b8b0-ad78fced5257 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:45.203374 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 170/348] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Oct 22 23:55:45 2022] POST /identity/v3/users => generated 246 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:55:45.204986 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1b96a-5a5e-4230-bdf7-a26f765ba7b3 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:45.220118 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 179/349] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:45.225428 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fb398-256f-4818-9b12-b172aabebf68 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:45.225721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fb398-256f-4818-9b12-b172aabebf68 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:45.225997 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fb398-256f-4818-9b12-b172aabebf68 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:45.227583 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 180/350] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:45.228185 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074fb398-256f-4818-9b12-b172aabebf68 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:45.334711 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:45.348832 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1b96a-5a5e-4230-bdf7-a26f765ba7b3 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:45.387534 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4815f-cd9d-4913-94ae-222ee42821cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:45.388160 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4815f-cd9d-4913-94ae-222ee42821cd None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:45.388444 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4815f-cd9d-4913-94ae-222ee42821cd None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:45.388742 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4815f-cd9d-4913-94ae-222ee42821cd None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:45.389464 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b4815f-cd9d-4913-94ae-222ee42821cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:45.390930 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b4815f-cd9d-4913-94ae-222ee42821cd None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:45.406434 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074fb398-256f-4818-9b12-b172aabebf68 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:45.424515 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 171/351] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Oct 22 23:55:45 2022] POST /identity/v3/users => generated 248 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:55:45.424515 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bce68c-4cc5-4a01-975a-17c56fb0ff9e None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:45.424515 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bce68c-4cc5-4a01-975a-17c56fb0ff9e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:45.424515 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bce68c-4cc5-4a01-975a-17c56fb0ff9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:45.437866 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-a2bce68c-4cc5-4a01-975a-17c56fb0ff9e None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:45.439236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a2bce68c-4cc5-4a01-975a-17c56fb0ff9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:45.452108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:45.457066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12b3d7-e977-4aad-b494-0d6b50ce2dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:45.457853 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12b3d7-e977-4aad-b494-0d6b50ce2dfa None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:45.458143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12b3d7-e977-4aad-b494-0d6b50ce2dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:45.458478 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12b3d7-e977-4aad-b494-0d6b50ce2dfa None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:45.459318 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad12b3d7-e977-4aad-b494-0d6b50ce2dfa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:45.461731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad12b3d7-e977-4aad-b494-0d6b50ce2dfa None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:45.490062 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 181/352] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Oct 22 23:55:45 2022] POST /identity/v3/users => generated 248 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:55:45.538856 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 172/353] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:45.851347 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12b3d7-e977-4aad-b494-0d6b50ce2dfa None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:45.945888 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300d5d6-ca4f-4527-b3eb-6623ef96d5bb None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:45.946273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300d5d6-ca4f-4527-b3eb-6623ef96d5bb None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:45.946626 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300d5d6-ca4f-4527-b3eb-6623ef96d5bb None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:45.948332 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 173/354] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 22 23:55:45.956520 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ce41d-874a-4ae8-87ea-baef1efe09b3 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:45.956520 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ce41d-874a-4ae8-87ea-baef1efe09b3 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:45.956520 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953ce41d-874a-4ae8-87ea-baef1efe09b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:45.986483 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12b3d7-e977-4aad-b494-0d6b50ce2dfa None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:45.988972 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-953ce41d-874a-4ae8-87ea-baef1efe09b3 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:45.990168 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-953ce41d-874a-4ae8-87ea-baef1efe09b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:46.008624 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:46.041490 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ffea40-2e7f-4b01-8a99-a70826521a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:46.042138 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ffea40-2e7f-4b01-8a99-a70826521a35 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:46.042420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ffea40-2e7f-4b01-8a99-a70826521a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:46.042693 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ffea40-2e7f-4b01-8a99-a70826521a35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:46.062155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ffea40-2e7f-4b01-8a99-a70826521a35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:46.064198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ffea40-2e7f-4b01-8a99-a70826521a35 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:46.079901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-13ffea40-2e7f-4b01-8a99-a70826521a35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:46.079901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:46.079901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:46.079901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:46.079901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:46.079901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:46.079901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:46.079901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:46.079901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:46.079901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:46.079901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:46.079901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:46.079901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:55:46.079901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:55:46.079901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:55:46.079901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:55:46.080894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:46.080894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:46.080894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:46.080894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:46.080894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:46.080894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:46.080894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:46.080894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:46.080894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:46.080894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:46.080894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:46.080894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:46.080894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:46.080894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:46.080894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:46.080894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:46.080894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:55:46.081733 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:55:46.081733 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:46.081733 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:46.081733 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 182/355] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:46.088377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ffea40-2e7f-4b01-8a99-a70826521a35 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:46.144600 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 174/356] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 22 23:55:46.253528 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ffea40-2e7f-4b01-8a99-a70826521a35 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:46.310592 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4db73d-a092-4deb-995a-d2240bf3d40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:46.311211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4db73d-a092-4deb-995a-d2240bf3d40c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:46.311495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4db73d-a092-4deb-995a-d2240bf3d40c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:46.311775 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4db73d-a092-4deb-995a-d2240bf3d40c None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:46.312423 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4db73d-a092-4deb-995a-d2240bf3d40c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:46.312758 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4db73d-a092-4deb-995a-d2240bf3d40c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:46.314280 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4db73d-a092-4deb-995a-d2240bf3d40c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:46.337709 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 183/357] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:55:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:55:46.343010 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ce41d-874a-4ae8-87ea-baef1efe09b3 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:46.442511 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:46.486724 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953ce41d-874a-4ae8-87ea-baef1efe09b3 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:46.540569 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7d9de-e83d-465b-95ee-d279876f8a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:46.541575 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7d9de-e83d-465b-95ee-d279876f8a45 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:46.542057 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7d9de-e83d-465b-95ee-d279876f8a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:46.542542 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7d9de-e83d-465b-95ee-d279876f8a45 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:46.543414 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7d9de-e83d-465b-95ee-d279876f8a45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:46.545077 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7d9de-e83d-465b-95ee-d279876f8a45 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:46.576197 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 175/358] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Oct 22 23:55:46 2022] POST /identity/v3/users => generated 249 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:55:46.653775 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4db73d-a092-4deb-995a-d2240bf3d40c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:46.685289 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:46.773519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4db73d-a092-4deb-995a-d2240bf3d40c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:46.829493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfeb25-72bf-4c5e-b1d9-8c43798e41d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:46.830202 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfeb25-72bf-4c5e-b1d9-8c43798e41d3 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:46.830489 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfeb25-72bf-4c5e-b1d9-8c43798e41d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:46.830765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfeb25-72bf-4c5e-b1d9-8c43798e41d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:46.831451 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfeb25-72bf-4c5e-b1d9-8c43798e41d3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:46.832836 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfeb25-72bf-4c5e-b1d9-8c43798e41d3 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:46.846481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.sql.core [None req-c2cfeb25-72bf-4c5e-b1d9-8c43798e41d3 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 22 23:55:46.846481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 22 23:55:46.846481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [parameters: {'id': 'c09b5b25aabd4292908126f8f20c9d19', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 22 23:55:46.846481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81381) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 22 23:55:46.854259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-c2cfeb25-72bf-4c5e-b1d9-8c43798e41d3 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 22 23:55:46.854259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:46.854259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 22 23:55:46.854259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 22 23:55:46.854259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 22 23:55:46.854259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 22 23:55:46.854259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 22 23:55:46.854259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result = self._query(query) Oct 22 23:55:46.854259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 22 23:55:46.854259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application conn.query(q) Oct 22 23:55:46.854259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 22 23:55:46.854259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 22 23:55:46.854259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 22 23:55:46.854259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result.read() Oct 22 23:55:46.854259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 22 23:55:46.855407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 22 23:55:46.855407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 22 23:55:46.855407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 22 23:55:46.855407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 22 23:55:46.855407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 22 23:55:46.855407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 22 23:55:46.855407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 22 23:55:46.855407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 22 23:55:46.855407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:46.855407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 22 23:55:46.855407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:46.855407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:46.855407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 22 23:55:46.855407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 22 23:55:46.855407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 22 23:55:46.855407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return ref.to_dict() Oct 22 23:55:46.855407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 22 23:55:46.855407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application next(self.gen) Oct 22 23:55:46.856344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 22 23:55:46.856344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application yield resource Oct 22 23:55:46.856344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 22 23:55:46.856344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application next(self.gen) Oct 22 23:55:46.856344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 22 23:55:46.856344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.session.rollback() Oct 22 23:55:46.856344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 22 23:55:46.856344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.force_reraise() Oct 22 23:55:46.856344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 22 23:55:46.856344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise self.value Oct 22 23:55:46.856344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 22 23:55:46.856344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 22 23:55:46.856344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 22 23:55:46.856344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application session.commit() Oct 22 23:55:46.856344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 22 23:55:46.856344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 22 23:55:46.856344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 22 23:55:46.857781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._prepare_impl() Oct 22 23:55:46.857781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 22 23:55:46.857781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.session.flush() Oct 22 23:55:46.857781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 22 23:55:46.857781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._flush(objects) Oct 22 23:55:46.857781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 22 23:55:46.857781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 22 23:55:46.857781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 22 23:55:46.857781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application compat.raise_( Oct 22 23:55:46.857781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 22 23:55:46.857781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception Oct 22 23:55:46.857781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 22 23:55:46.857781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application flush_context.execute() Oct 22 23:55:46.857781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 22 23:55:46.857781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rec.execute(self) Oct 22 23:55:46.857781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 22 23:55:46.857781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 22 23:55:46.858684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 22 23:55:46.858684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 22 23:55:46.858684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 22 23:55:46.858684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 22 23:55:46.858684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 22 23:55:46.858684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 22 23:55:46.858684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 22 23:55:46.858684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 22 23:55:46.858684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 22 23:55:46.858684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 22 23:55:46.858684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 22 23:55:46.858684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 22 23:55:46.858684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 22 23:55:46.858684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 22 23:55:46.858684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 22 23:55:46.858684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception Oct 22 23:55:46.859493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 22 23:55:46.859493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 22 23:55:46.859493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 22 23:55:46.859493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 22 23:55:46.859493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 22 23:55:46.859493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result = self._query(query) Oct 22 23:55:46.859493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 22 23:55:46.859493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application conn.query(q) Oct 22 23:55:46.859493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 22 23:55:46.859493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 22 23:55:46.859493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 22 23:55:46.859493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result.read() Oct 22 23:55:46.859493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 22 23:55:46.859493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 22 23:55:46.859493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 22 23:55:46.859493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 22 23:55:46.860365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 22 23:55:46.860365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 22 23:55:46.860365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 22 23:55:46.860365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 22 23:55:46.860365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 22 23:55:46.860365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 22 23:55:46.860365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [parameters: {'id': 'c09b5b25aabd4292908126f8f20c9d19', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 22 23:55:46.860365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 22 23:55:46.860365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:46.860365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:55:46.860365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:46.860365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:46.860365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:46.860365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:46.860365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:46.860365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:46.863155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:46.863155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:46.863155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:46.863155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:46.863155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:46.863155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:46.863155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 22 23:55:46.863155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 22 23:55:46.863155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:46.863155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:46.863155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 22 23:55:46.863155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 22 23:55:46.863155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 22 23:55:46.863155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 22 23:55:46.863155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 22 23:55:46.863155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:46.865183 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 184/359] 10.222.0.13 () {60 vars in 1131 bytes} [Sat Oct 22 23:55:46 2022] POST /identity/v3/roles => generated 144 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:46.865326 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7d9de-e83d-465b-95ee-d279876f8a45 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:46.987890 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7d9de-e83d-465b-95ee-d279876f8a45 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:47.016189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:47.025632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a551c-e76f-4b99-8008-b01eae4c2038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:47.026203 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a551c-e76f-4b99-8008-b01eae4c2038 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:47.026486 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a551c-e76f-4b99-8008-b01eae4c2038 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:47.026759 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a551c-e76f-4b99-8008-b01eae4c2038 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:47.038397 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a551c-e76f-4b99-8008-b01eae4c2038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:47.043819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-e91a551c-e76f-4b99-8008-b01eae4c2038 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 22 23:55:47.043819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:47.043819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:47.043819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:47.043819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:47.043819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:47.043819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:47.043819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:47.043819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:47.043819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:47.043819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:47.043819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:47.043819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:47.043819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:47.043819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:47.043819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:47.044770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:47.044770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:47.044770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:47.044770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:47.044770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:47.044770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:47.044770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:47.044770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:47.044770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:47.044770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:47.044770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:47.044770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:47.044770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:47.044770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:47.044770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:47.044770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:47.045628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:47.045628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:47.045628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:47.045628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:47.045628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:47.045628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:47.045628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:47.045628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:47.045628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:47.045628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:47.045628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 22 23:55:47.045628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:47.045628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 176/360] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Oct 22 23:55:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:47.051637 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfeb25-72bf-4c5e-b1d9-8c43798e41d3 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:47.166351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:47.170779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfeb25-72bf-4c5e-b1d9-8c43798e41d3 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:47.207089 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32b834-e0d5-49fe-8ec9-30399673d68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:47.207707 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32b834-e0d5-49fe-8ec9-30399673d68a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:47.207993 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32b834-e0d5-49fe-8ec9-30399673d68a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:47.208277 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32b834-e0d5-49fe-8ec9-30399673d68a None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:47.208987 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32b834-e0d5-49fe-8ec9-30399673d68a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:47.209294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32b834-e0d5-49fe-8ec9-30399673d68a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:47.210653 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32b834-e0d5-49fe-8ec9-30399673d68a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:47.218786 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 185/361] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:55:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:47.973630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1c205-434b-4265-95f1-2dc49f622c33 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:47.974046 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1c205-434b-4265-95f1-2dc49f622c33 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:47.974220 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1c205-434b-4265-95f1-2dc49f622c33 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:47.975986 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 177/362] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:47.982456 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85afb7ac-d0be-471e-8fa2-d193c8ff5465 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:47.982752 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85afb7ac-d0be-471e-8fa2-d193c8ff5465 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:47.983009 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85afb7ac-d0be-471e-8fa2-d193c8ff5465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:48.003082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-85afb7ac-d0be-471e-8fa2-d193c8ff5465 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:48.004278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-85afb7ac-d0be-471e-8fa2-d193c8ff5465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:48.073095 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 186/363] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:48.080887 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904018cf-b976-4eee-ac37-b3b2ed0f009c None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:48.081746 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904018cf-b976-4eee-ac37-b3b2ed0f009c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:48.082240 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904018cf-b976-4eee-ac37-b3b2ed0f009c None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:48.085001 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 178/364] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:48.089462 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23700a2-4853-40ff-9d0b-52e05a508eb7 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:48.089992 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23700a2-4853-40ff-9d0b-52e05a508eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:48.090479 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23700a2-4853-40ff-9d0b-52e05a508eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:48.117657 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-f23700a2-4853-40ff-9d0b-52e05a508eb7 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:48.118876 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f23700a2-4853-40ff-9d0b-52e05a508eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:48.164207 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d24f4-e59c-4dfe-bb6f-16c816fd26c6 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:48.164543 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d24f4-e59c-4dfe-bb6f-16c816fd26c6 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:48.164790 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d24f4-e59c-4dfe-bb6f-16c816fd26c6 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:48.166141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 179/365] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 22 23:55:48.175911 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193d513-1543-4acc-95ff-b24f2231d026 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:48.176631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193d513-1543-4acc-95ff-b24f2231d026 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:48.177213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3193d513-1543-4acc-95ff-b24f2231d026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:48.198563 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-3193d513-1543-4acc-95ff-b24f2231d026 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:48.200415 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-3193d513-1543-4acc-95ff-b24f2231d026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:48.243096 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 187/366] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:48.323553 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 180/367] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:48.396960 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23700a2-4853-40ff-9d0b-52e05a508eb7 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:48.406020 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:48.526489 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23700a2-4853-40ff-9d0b-52e05a508eb7 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:48.576473 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027b53c-e9cd-4e2f-aa29-3deaedf7a43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:48.577515 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027b53c-e9cd-4e2f-aa29-3deaedf7a43e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:48.578004 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027b53c-e9cd-4e2f-aa29-3deaedf7a43e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:48.578494 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027b53c-e9cd-4e2f-aa29-3deaedf7a43e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:48.597361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2027b53c-e9cd-4e2f-aa29-3deaedf7a43e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:48.602712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-2027b53c-e9cd-4e2f-aa29-3deaedf7a43e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:55:48.602712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:48.602712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:48.602712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:48.602712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:48.602712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:48.602712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:48.602712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:48.602712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:48.602712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:48.602712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:48.602712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:48.602712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:48.602712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:48.602712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:48.602712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:48.603593 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:48.603593 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:48.603593 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:48.603593 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:48.603593 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:48.603593 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:48.603593 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:48.603593 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:48.603593 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:48.603593 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:48.603593 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:48.603593 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:48.603593 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:48.603593 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:48.603593 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:48.603593 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:48.604389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:48.604389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:48.604389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:48.604389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:48.604389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:48.604389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:48.604389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:48.604389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:48.604389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:48.604389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:48.604389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:55:48.604389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:48.606360 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 188/368] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Oct 22 23:55:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:48.608829 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193d513-1543-4acc-95ff-b24f2231d026 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:48.613143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027b53c-e9cd-4e2f-aa29-3deaedf7a43e None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:48.723992 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:48.763963 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3193d513-1543-4acc-95ff-b24f2231d026 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:48.764525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027b53c-e9cd-4e2f-aa29-3deaedf7a43e None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:48.784187 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:48.807386 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff99824-2db9-4a99-9a59-aa188db67d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:48.807831 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14885e26-2700-4d9d-a5e3-a4b0a9de654f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:48.808418 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff99824-2db9-4a99-9a59-aa188db67d0e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:48.808673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14885e26-2700-4d9d-a5e3-a4b0a9de654f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:48.809031 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff99824-2db9-4a99-9a59-aa188db67d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:48.809146 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14885e26-2700-4d9d-a5e3-a4b0a9de654f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:48.809355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14885e26-2700-4d9d-a5e3-a4b0a9de654f None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:48.809697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff99824-2db9-4a99-9a59-aa188db67d0e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:48.810160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14885e26-2700-4d9d-a5e3-a4b0a9de654f None admin] RBAC: Adding query filter params (name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:48.810602 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14885e26-2700-4d9d-a5e3-a4b0a9de654f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:48.812303 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14885e26-2700-4d9d-a5e3-a4b0a9de654f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:48.821874 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff99824-2db9-4a99-9a59-aa188db67d0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:48.822609 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 189/369] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Oct 22 23:55:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:55:48.823632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14885e26-2700-4d9d-a5e3-a4b0a9de654f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:48.828724 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-fff99824-2db9-4a99-9a59-aa188db67d0e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:55:48.828724 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:48.828724 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:48.828724 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:48.828724 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:48.828724 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:48.828724 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:48.828724 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:48.828724 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:48.828724 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:48.828724 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:48.828724 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:48.828724 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:48.828724 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:48.828724 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:48.828724 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:48.830254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:48.830254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:48.830254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:48.830254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:48.830254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:48.830254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:48.830254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:48.830254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:48.830254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:48.830254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:48.830254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:48.830254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:48.830254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:48.830254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:48.830254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:48.830254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:48.831067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:48.831067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:48.831067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:48.831067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:48.831067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:48.831067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:48.831067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:48.831067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:48.831067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:48.831067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:48.831067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:55:48.831067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:48.833589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 181/370] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Oct 22 23:55:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:48.834593 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff99824-2db9-4a99-9a59-aa188db67d0e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:48.954136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14885e26-2700-4d9d-a5e3-a4b0a9de654f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:48.966518 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff99824-2db9-4a99-9a59-aa188db67d0e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:48.987070 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:48.997398 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d0028-52ee-4c9d-aa67-1870e3cc7138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:48.998247 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d0028-52ee-4c9d-aa67-1870e3cc7138 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:48.998721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d0028-52ee-4c9d-aa67-1870e3cc7138 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:48.999162 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d0028-52ee-4c9d-aa67-1870e3cc7138 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:49.013414 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d0028-52ee-4c9d-aa67-1870e3cc7138 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:49.013831 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f9146-a62f-4f31-90ab-2a5e4d980a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:49.014539 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f9146-a62f-4f31-90ab-2a5e4d980a6c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:49.014929 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f9146-a62f-4f31-90ab-2a5e4d980a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:49.015282 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f9146-a62f-4f31-90ab-2a5e4d980a6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:49.020789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-9f1d0028-52ee-4c9d-aa67-1870e3cc7138 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:55:49.020789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:49.020789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:49.020789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:49.020789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:49.020789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:49.020789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:49.020789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:49.020789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:49.020789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:49.020789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:49.020789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:49.020789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:49.020789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:49.020789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:49.020789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:49.021673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:49.021673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:49.021673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:49.021673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:49.021673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:49.021673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:49.021673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:49.021673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:49.021673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:49.021673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:49.021673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:49.021673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:49.021673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:49.021673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:49.021673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:49.021673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:49.022499 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:49.022499 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:49.022499 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:49.022499 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:49.022499 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:49.022499 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:49.022499 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:49.022499 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:49.022499 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:49.022499 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:49.022499 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:55:49.022499 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:49.026281 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f9146-a62f-4f31-90ab-2a5e4d980a6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:49.028002 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f9146-a62f-4f31-90ab-2a5e4d980a6c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:49.029479 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 190/371] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Oct 22 23:55:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:49.030672 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d0028-52ee-4c9d-aa67-1870e3cc7138 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:49.051096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-8f3f9146-a62f-4f31-90ab-2a5e4d980a6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:49.051096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:49.051096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:49.051096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:49.051096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:49.051096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:49.051096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:49.051096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:49.051096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:49.051096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:49.051096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:49.051096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:49.051096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:55:49.051096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:55:49.051096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:55:49.051096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:55:49.052067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:49.052067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:49.052067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:49.052067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:49.052067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:49.052067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:49.052067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:49.052067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:49.052067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:49.052067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:49.052067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:49.052067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:49.052067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:49.052067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:49.052067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:49.052067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:49.052067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:55:49.052984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:55:49.052984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:49.052984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:49.054194 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 182/372] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:49.059192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f9146-a62f-4f31-90ab-2a5e4d980a6c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:49.099501 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:49.162382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d0028-52ee-4c9d-aa67-1870e3cc7138 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:49.199642 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f9146-a62f-4f31-90ab-2a5e4d980a6c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:49.201629 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb9247-90c2-499a-8bad-df498cb8cf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:49.203248 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb9247-90c2-499a-8bad-df498cb8cf17 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:49.203630 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb9247-90c2-499a-8bad-df498cb8cf17 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:49.204063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb9247-90c2-499a-8bad-df498cb8cf17 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:49.204933 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb9247-90c2-499a-8bad-df498cb8cf17 None admin] RBAC: Adding query filter params (name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:49.205338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb9247-90c2-499a-8bad-df498cb8cf17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:49.206799 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb9247-90c2-499a-8bad-df498cb8cf17 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:49.222068 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 191/373] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Oct 22 23:55:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:49.222068 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb9247-90c2-499a-8bad-df498cb8cf17 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:49.242136 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fa014-2fe9-47a8-a96b-af0d2f64c043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:49.243109 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fa014-2fe9-47a8-a96b-af0d2f64c043 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:49.243596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fa014-2fe9-47a8-a96b-af0d2f64c043 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:49.244072 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fa014-2fe9-47a8-a96b-af0d2f64c043 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:49.245011 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14fa014-2fe9-47a8-a96b-af0d2f64c043 None admin] RBAC: Adding query filter params (name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:49.245520 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14fa014-2fe9-47a8-a96b-af0d2f64c043 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:49.247128 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14fa014-2fe9-47a8-a96b-af0d2f64c043 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:49.255658 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 183/374] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Oct 22 23:55:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:49.256780 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fa014-2fe9-47a8-a96b-af0d2f64c043 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:49.285114 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:49.360227 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb9247-90c2-499a-8bad-df498cb8cf17 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:49.362534 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:49.373026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fa014-2fe9-47a8-a96b-af0d2f64c043 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:49.405102 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30845024-1bf4-42f3-a14a-4b32f9ad103f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:49.405715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30845024-1bf4-42f3-a14a-4b32f9ad103f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:49.405997 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30845024-1bf4-42f3-a14a-4b32f9ad103f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:49.406271 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30845024-1bf4-42f3-a14a-4b32f9ad103f None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:49.406904 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30845024-1bf4-42f3-a14a-4b32f9ad103f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:49.407467 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30845024-1bf4-42f3-a14a-4b32f9ad103f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:49.408995 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30845024-1bf4-42f3-a14a-4b32f9ad103f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:49.409577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef9608-32e6-4473-864b-2f81b61edeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:49.410420 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef9608-32e6-4473-864b-2f81b61edeb4 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:49.410898 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef9608-32e6-4473-864b-2f81b61edeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:49.411382 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef9608-32e6-4473-864b-2f81b61edeb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:49.427685 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 192/375] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:49.427685 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30845024-1bf4-42f3-a14a-4b32f9ad103f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:49.427950 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef9608-32e6-4473-864b-2f81b61edeb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:49.427950 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef9608-32e6-4473-864b-2f81b61edeb4 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:49.441475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-63ef9608-32e6-4473-864b-2f81b61edeb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:49.441475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:49.441475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:49.441475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:49.441475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:49.441475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:49.441475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:49.441475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:49.441475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:49.441475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:49.441475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:49.441475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:49.441475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:55:49.441475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:55:49.441475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:55:49.441475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:55:49.442719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:49.442719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:49.442719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:49.442719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:49.442719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:49.442719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:49.442719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:49.442719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:49.442719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:49.442719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:49.442719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:49.442719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:49.442719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:49.442719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:49.442719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:49.442719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:49.442719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:55:49.443593 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:55:49.443593 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:49.443593 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:49.443593 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 184/376] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:49.448424 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ed173-5289-409f-8b10-54390c907b00 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:49.448424 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ed173-5289-409f-8b10-54390c907b00 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:49.448424 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ed173-5289-409f-8b10-54390c907b00 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:49.450487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 185/377] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:49.451210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ed173-5289-409f-8b10-54390c907b00 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:49.531433 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30845024-1bf4-42f3-a14a-4b32f9ad103f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:49.543308 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:49.561598 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ed173-5289-409f-8b10-54390c907b00 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:49.569460 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569aa7a-bdbf-43cb-a2bf-d1aca9dde32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:49.570160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569aa7a-bdbf-43cb-a2bf-d1aca9dde32f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:49.570448 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569aa7a-bdbf-43cb-a2bf-d1aca9dde32f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:49.570727 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569aa7a-bdbf-43cb-a2bf-d1aca9dde32f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:49.581599 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c569aa7a-bdbf-43cb-a2bf-d1aca9dde32f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:49.583043 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c569aa7a-bdbf-43cb-a2bf-d1aca9dde32f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:49.597319 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a1673-9634-456c-8ce9-c50c4c6d44ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:49.597944 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a1673-9634-456c-8ce9-c50c4c6d44ec None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:49.598220 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a1673-9634-456c-8ce9-c50c4c6d44ec None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:49.598489 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a1673-9634-456c-8ce9-c50c4c6d44ec None admin] PATH_INFO: `/v3/users/nova` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:49.601330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-c569aa7a-bdbf-43cb-a2bf-d1aca9dde32f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:49.601330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:49.601330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:49.601330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:49.601330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:49.601330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:49.601330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:49.601330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:49.601330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:49.601330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:49.601330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:49.601330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:49.601330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:55:49.601330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:55:49.601330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:55:49.601330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:55:49.602203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:49.602203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:49.602203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:49.602203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:49.602203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:49.602203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:49.602203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:49.602203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:49.602203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:49.602203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:49.602203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:49.602203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:49.602203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:49.602203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:49.602203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:49.602203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:49.602203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:55:49.603059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:55:49.603059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:49.603059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:49.606969 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 193/378] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:49.607741 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569aa7a-bdbf-43cb-a2bf-d1aca9dde32f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:49.608037 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73a1673-9634-456c-8ce9-c50c4c6d44ec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:49.609483 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73a1673-9634-456c-8ce9-c50c4c6d44ec None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:49.622487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-a73a1673-9634-456c-8ce9-c50c4c6d44ec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 22 23:55:49.622487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:49.622487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:49.622487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:49.622487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:49.622487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:49.622487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:49.622487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:49.622487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:49.622487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:49.622487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:49.622487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:49.622487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:55:49.622487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:55:49.622487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:55:49.622487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:55:49.623446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:49.623446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:49.623446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:55:49.623446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:55:49.623446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:55:49.623446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:55:49.623446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:49.623446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:49.623446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:49.623446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:49.623446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:49.623446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:49.623446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:49.623446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:49.623446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:49.623446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:49.623446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:49.623446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:49.624359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:49.624359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:49.624359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:55:49.624359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:55:49.624359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:55:49.624359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:55:49.624359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:55:49.624359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:55:49.624359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 22 23:55:49.624359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:49.624359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 186/379] 10.222.0.13 () {56 vars in 1093 bytes} [Sat Oct 22 23:55:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:49.625484 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de00722-a0d8-477f-9b1b-96736dee134e None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:49.625484 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de00722-a0d8-477f-9b1b-96736dee134e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:49.625484 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de00722-a0d8-477f-9b1b-96736dee134e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:49.633134 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:49.645867 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-0de00722-a0d8-477f-9b1b-96736dee134e None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:49.647304 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-0de00722-a0d8-477f-9b1b-96736dee134e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:49.720831 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569aa7a-bdbf-43cb-a2bf-d1aca9dde32f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:49.721188 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 187/380] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:49.721960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de00722-a0d8-477f-9b1b-96736dee134e None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:49.778592 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a5fae-207a-48a4-8368-2140dbe75767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:49.779575 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a5fae-207a-48a4-8368-2140dbe75767 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:49.780054 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a5fae-207a-48a4-8368-2140dbe75767 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:49.780540 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a5fae-207a-48a4-8368-2140dbe75767 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:49.781970 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a5fae-207a-48a4-8368-2140dbe75767 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:49.783288 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a5fae-207a-48a4-8368-2140dbe75767 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:49.785528 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a5fae-207a-48a4-8368-2140dbe75767 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:49.802905 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 194/381] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:49.804005 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a5fae-207a-48a4-8368-2140dbe75767 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:49.835582 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:49.846029 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de00722-a0d8-477f-9b1b-96736dee134e None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:49.890357 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba67543-b1a6-45a4-bf1c-7214d169119e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:49.890917 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba67543-b1a6-45a4-bf1c-7214d169119e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:49.891204 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba67543-b1a6-45a4-bf1c-7214d169119e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:49.891718 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba67543-b1a6-45a4-bf1c-7214d169119e None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:49.892355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba67543-b1a6-45a4-bf1c-7214d169119e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:49.892768 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba67543-b1a6-45a4-bf1c-7214d169119e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:49.894140 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba67543-b1a6-45a4-bf1c-7214d169119e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:49.896046 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:49.918592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 188/382] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:49.919496 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba67543-b1a6-45a4-bf1c-7214d169119e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:49.940365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a5fae-207a-48a4-8368-2140dbe75767 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:49.989879 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b118f-4518-4dab-be8a-04737c54bc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:49.990843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b118f-4518-4dab-be8a-04737c54bc3a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:49.991324 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b118f-4518-4dab-be8a-04737c54bc3a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:49.992088 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b118f-4518-4dab-be8a-04737c54bc3a None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:49.993746 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b118f-4518-4dab-be8a-04737c54bc3a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:49.994327 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b118f-4518-4dab-be8a-04737c54bc3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:49.996528 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b118f-4518-4dab-be8a-04737c54bc3a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:50.030497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba67543-b1a6-45a4-bf1c-7214d169119e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:50.046875 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 195/383] 10.222.0.13 () {56 vars in 1133 bytes} [Sat Oct 22 23:55:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:50.046875 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b118f-4518-4dab-be8a-04737c54bc3a None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:50.077411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3b8bc-563b-40d0-9111-161174055b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:50.078042 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3b8bc-563b-40d0-9111-161174055b14 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:50.078327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3b8bc-563b-40d0-9111-161174055b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:50.078605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3b8bc-563b-40d0-9111-161174055b14 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:50.086810 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3b8bc-563b-40d0-9111-161174055b14 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:50.088259 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3b8bc-563b-40d0-9111-161174055b14 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:50.103701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-72f3b8bc-563b-40d0-9111-161174055b14 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 22 23:55:50.103701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:50.103701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:50.103701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:50.103701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:50.103701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:50.103701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:50.103701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:50.103701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:50.103701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:50.103701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:50.103701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:50.103701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:55:50.103701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:55:50.103701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:55:50.103701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:55:50.105317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:50.105317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:50.105317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:55:50.105317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:55:50.105317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:55:50.105317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:55:50.105317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:50.105317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:50.105317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:50.105317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:50.105317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:50.105317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:50.105317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:50.105317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:50.105317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:50.105317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:50.105317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:50.105317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:50.106403 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:50.106403 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:50.106403 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:55:50.106403 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:55:50.106403 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:55:50.106403 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:55:50.106403 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:55:50.106403 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:55:50.106403 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 22 23:55:50.106403 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:50.106403 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 189/384] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:50.106403 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3b8bc-563b-40d0-9111-161174055b14 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:50.144209 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:50.162936 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:50.165016 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b118f-4518-4dab-be8a-04737c54bc3a None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:50.199023 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bf6d5-f6f8-4a78-9e53-6cbf5ca7168f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:50.199804 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bf6d5-f6f8-4a78-9e53-6cbf5ca7168f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:50.200421 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bf6d5-f6f8-4a78-9e53-6cbf5ca7168f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:50.200752 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bf6d5-f6f8-4a78-9e53-6cbf5ca7168f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:50.208183 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994bf6d5-f6f8-4a78-9e53-6cbf5ca7168f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:50.208723 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3b8bc-563b-40d0-9111-161174055b14 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:50.209667 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994bf6d5-f6f8-4a78-9e53-6cbf5ca7168f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:50.223181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-994bf6d5-f6f8-4a78-9e53-6cbf5ca7168f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 22 23:55:50.223181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:50.223181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:50.223181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:50.223181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:50.223181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:50.223181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:50.223181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:50.223181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:50.223181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:50.223181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:50.223181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:50.223181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:55:50.223181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:55:50.223181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:55:50.223181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:55:50.224063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:50.224063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:50.224063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:55:50.224063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:55:50.224063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:55:50.224063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:55:50.224063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:50.224063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:50.224063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:50.224063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:50.224063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:50.224063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:50.224063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:50.224063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:50.224063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:50.224063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:50.224063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:50.224063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:50.224980 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:50.224980 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:50.224980 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:55:50.224980 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:55:50.224980 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:55:50.224980 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:55:50.224980 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:55:50.224980 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:55:50.224980 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 22 23:55:50.224980 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:50.224980 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 196/385] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:50.226683 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bf6d5-f6f8-4a78-9e53-6cbf5ca7168f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:50.242319 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70922030-ef70-4f13-a322-9cfb37005db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:50.242954 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70922030-ef70-4f13-a322-9cfb37005db5 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:50.243237 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70922030-ef70-4f13-a322-9cfb37005db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:50.243515 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70922030-ef70-4f13-a322-9cfb37005db5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:50.254036 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70922030-ef70-4f13-a322-9cfb37005db5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:50.259355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-70922030-ef70-4f13-a322-9cfb37005db5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:55:50.259355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:50.259355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:50.259355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:50.259355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:50.259355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:50.259355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:50.259355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:50.259355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:50.259355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:50.259355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:50.259355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:50.259355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:50.259355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:50.259355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:50.259355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:50.260257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:50.260257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:50.260257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:50.260257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:50.260257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:50.260257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:50.260257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:50.260257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:50.260257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:50.260257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:50.260257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:50.260257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:50.260257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:50.260257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:50.260257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:50.260257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:50.261391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:50.261391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:50.261391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:50.261391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:50.261391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:50.261391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:50.261391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:50.261391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:50.261391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:50.261391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:50.261391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:55:50.261391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:50.261391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 190/386] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Oct 22 23:55:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:50.261391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70922030-ef70-4f13-a322-9cfb37005db5 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:50.325082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bf6d5-f6f8-4a78-9e53-6cbf5ca7168f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:50.358551 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107e6d6-c6ad-4081-ab3a-996331eb5da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:50.359172 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107e6d6-c6ad-4081-ab3a-996331eb5da6 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:50.359453 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107e6d6-c6ad-4081-ab3a-996331eb5da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:50.359728 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107e6d6-c6ad-4081-ab3a-996331eb5da6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:50.367067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70922030-ef70-4f13-a322-9cfb37005db5 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:50.368005 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:50.370653 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107e6d6-c6ad-4081-ab3a-996331eb5da6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:50.372404 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107e6d6-c6ad-4081-ab3a-996331eb5da6 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:50.382022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:50.387838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-7107e6d6-c6ad-4081-ab3a-996331eb5da6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:50.387838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:50.387838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:50.387838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:50.387838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:50.387838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:50.387838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:50.387838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:50.387838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:50.387838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:50.387838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:50.387838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:50.387838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:55:50.387838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:55:50.387838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:55:50.387838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:55:50.393614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:50.393614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:50.393614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:50.393614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:50.393614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:50.393614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:50.393614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:50.393614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:50.393614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:50.393614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:50.393614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:50.393614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:50.393614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:50.393614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:50.393614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:50.393614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:50.393614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:55:50.394715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:55:50.394715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:50.394715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:50.394715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 197/387] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:50.394715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f4505-0858-4b20-9ec8-5b7e1948a336 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:50.394715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f4505-0858-4b20-9ec8-5b7e1948a336 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:50.394715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f4505-0858-4b20-9ec8-5b7e1948a336 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:50.394715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 198/388] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:50.394715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f4505-0858-4b20-9ec8-5b7e1948a336 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:50.410411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc99393-527f-48dc-ac00-182d71dab9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:50.411009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc99393-527f-48dc-ac00-182d71dab9b5 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:50.411774 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc99393-527f-48dc-ac00-182d71dab9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:50.412052 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc99393-527f-48dc-ac00-182d71dab9b5 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:50.412825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc99393-527f-48dc-ac00-182d71dab9b5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:50.413125 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc99393-527f-48dc-ac00-182d71dab9b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:50.414531 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc99393-527f-48dc-ac00-182d71dab9b5 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:50.450458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 191/389] 10.222.0.13 () {56 vars in 1137 bytes} [Sat Oct 22 23:55:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:50.451500 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc99393-527f-48dc-ac00-182d71dab9b5 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:50.498678 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f4505-0858-4b20-9ec8-5b7e1948a336 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:50.532610 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f55ed4-ce3d-4817-ba49-b9964554eb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:50.533214 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f55ed4-ce3d-4817-ba49-b9964554eb0c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:50.533500 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f55ed4-ce3d-4817-ba49-b9964554eb0c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:50.533779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f55ed4-ce3d-4817-ba49-b9964554eb0c None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:50.534509 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f55ed4-ce3d-4817-ba49-b9964554eb0c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:50.534822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f55ed4-ce3d-4817-ba49-b9964554eb0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:50.536236 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f55ed4-ce3d-4817-ba49-b9964554eb0c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:50.549725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc99393-527f-48dc-ac00-182d71dab9b5 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:50.553176 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 199/390] 10.222.0.13 () {56 vars in 1137 bytes} [Sat Oct 22 23:55:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:50.554017 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f55ed4-ce3d-4817-ba49-b9964554eb0c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:50.584295 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c5086-dcfb-402f-b974-6476e6ca3f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:50.584926 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c5086-dcfb-402f-b974-6476e6ca3f77 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:50.585208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c5086-dcfb-402f-b974-6476e6ca3f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:50.585483 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c5086-dcfb-402f-b974-6476e6ca3f77 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:50.586142 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37c5086-dcfb-402f-b974-6476e6ca3f77 None admin] RBAC: Adding query filter params (name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:50.586446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37c5086-dcfb-402f-b974-6476e6ca3f77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:50.587823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37c5086-dcfb-402f-b974-6476e6ca3f77 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:50.596007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 192/391] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Oct 22 23:55:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:50.597179 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80074e45-2812-4744-95d6-187ad22050b5 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:50.597474 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80074e45-2812-4744-95d6-187ad22050b5 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:50.597750 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80074e45-2812-4744-95d6-187ad22050b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:50.607391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:50.616322 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-80074e45-2812-4744-95d6-187ad22050b5 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:50.617533 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-80074e45-2812-4744-95d6-187ad22050b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:50.643394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:50.657653 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f55ed4-ce3d-4817-ba49-b9964554eb0c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:50.684361 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 193/392] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:50.693011 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1201f4e-d5bb-4cd9-a8c9-37ec52b23aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:50.693371 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80074e45-2812-4744-95d6-187ad22050b5 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:50.693623 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1201f4e-d5bb-4cd9-a8c9-37ec52b23aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:50.693908 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1201f4e-d5bb-4cd9-a8c9-37ec52b23aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:50.694187 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1201f4e-d5bb-4cd9-a8c9-37ec52b23aa2 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:50.694821 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1201f4e-d5bb-4cd9-a8c9-37ec52b23aa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:50.695125 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1201f4e-d5bb-4cd9-a8c9-37ec52b23aa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:50.696515 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1201f4e-d5bb-4cd9-a8c9-37ec52b23aa2 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:50.708316 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 200/393] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:50.709243 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1201f4e-d5bb-4cd9-a8c9-37ec52b23aa2 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:50.803540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80074e45-2812-4744-95d6-187ad22050b5 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:50.818205 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1201f4e-d5bb-4cd9-a8c9-37ec52b23aa2 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:50.829541 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:50.845122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e7309-69a9-4450-88b9-b84d8c5a4fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:50.845919 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e7309-69a9-4450-88b9-b84d8c5a4fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:50.846532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e7309-69a9-4450-88b9-b84d8c5a4fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:50.846814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e7309-69a9-4450-88b9-b84d8c5a4fe8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:50.853715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2470d5b-8b1e-430c-8f90-2dfca0a43d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:50.854331 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2470d5b-8b1e-430c-8f90-2dfca0a43d72 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:50.854613 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2470d5b-8b1e-430c-8f90-2dfca0a43d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:50.854887 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2470d5b-8b1e-430c-8f90-2dfca0a43d72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:50.858378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e7309-69a9-4450-88b9-b84d8c5a4fe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:50.859834 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e7309-69a9-4450-88b9-b84d8c5a4fe8 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:50.866284 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2470d5b-8b1e-430c-8f90-2dfca0a43d72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:50.867715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2470d5b-8b1e-430c-8f90-2dfca0a43d72 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:50.874979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-659e7309-69a9-4450-88b9-b84d8c5a4fe8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:50.874979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:50.874979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:50.874979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:50.874979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:50.874979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:50.874979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:50.874979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:50.874979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:50.874979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:50.874979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:50.874979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:50.874979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:55:50.874979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:55:50.874979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:55:50.874979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:55:50.875877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:50.875877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:50.875877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:50.875877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:50.875877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:50.875877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:50.875877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:50.875877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:50.875877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:50.875877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:50.875877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:50.875877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:50.875877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:50.875877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:50.875877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:50.875877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:50.875877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:55:50.876780 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:55:50.876780 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:50.876780 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:50.876780 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 194/394] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:50.881455 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e7309-69a9-4450-88b9-b84d8c5a4fe8 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:50.886320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-f2470d5b-8b1e-430c-8f90-2dfca0a43d72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:50.886320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:50.886320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:50.886320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:50.886320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:50.886320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:50.886320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:50.886320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:50.886320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:50.886320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:50.886320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:50.886320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:50.886320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:55:50.886320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:55:50.886320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:55:50.886320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:55:50.887216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:50.887216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:50.887216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:50.887216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:50.887216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:50.887216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:50.887216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:50.887216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:50.887216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:50.887216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:50.887216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:50.887216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:50.887216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:50.887216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:50.887216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:50.887216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:50.887216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:55:50.888086 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:55:50.888086 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:50.888086 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:50.888086 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 201/395] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:50.888863 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2470d5b-8b1e-430c-8f90-2dfca0a43d72 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:50.903454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:50.991414 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e7309-69a9-4450-88b9-b84d8c5a4fe8 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:50.992001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2470d5b-8b1e-430c-8f90-2dfca0a43d72 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:51.024355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c0128-2b23-4c34-99ff-6fe26ab6d7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:51.025003 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c0128-2b23-4c34-99ff-6fe26ab6d7d5 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:51.025204 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463d9ff-bf82-40e4-8d74-13ce1755dd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:51.025330 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c0128-2b23-4c34-99ff-6fe26ab6d7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:51.025546 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c0128-2b23-4c34-99ff-6fe26ab6d7d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:51.025831 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463d9ff-bf82-40e4-8d74-13ce1755dd2c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:51.026120 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463d9ff-bf82-40e4-8d74-13ce1755dd2c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:51.026403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463d9ff-bf82-40e4-8d74-13ce1755dd2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:51.032918 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9463d9ff-bf82-40e4-8d74-13ce1755dd2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:51.034362 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9463d9ff-bf82-40e4-8d74-13ce1755dd2c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:51.035473 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c0128-2b23-4c34-99ff-6fe26ab6d7d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:51.036952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c0128-2b23-4c34-99ff-6fe26ab6d7d5 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:51.045716 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-9463d9ff-bf82-40e4-8d74-13ce1755dd2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:55:51.045716 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:51.045716 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:51.045716 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:51.045716 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:51.045716 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:51.045716 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:51.045716 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:51.045716 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:51.045716 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:51.045716 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:51.045716 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:51.045716 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:55:51.045716 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:55:51.045716 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:55:51.046718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:55:51.046718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:51.046718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:51.046718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:51.046718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:51.046718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:51.046718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:51.046718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:51.046718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:51.046718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:51.046718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:51.046718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:51.046718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:51.046718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:51.046718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:51.046718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:51.046718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:51.048160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:55:51.048160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:55:51.048160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:55:51.048160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:55:51.048160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:55:51.048160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:55:51.048160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:55:51.048160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:51.048160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 202/396] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:55:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:51.048160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463d9ff-bf82-40e4-8d74-13ce1755dd2c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:51.054747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-3e2c0128-2b23-4c34-99ff-6fe26ab6d7d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:51.054747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:51.054747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:51.054747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:51.054747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:51.054747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:51.054747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:51.054747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:51.054747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:51.054747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:51.054747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:51.054747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:51.054747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:55:51.054747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:55:51.054747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:55:51.054747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:55:51.055858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:51.055858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:51.055858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:51.055858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:51.055858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:51.055858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:51.055858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:51.055858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:51.055858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:51.055858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:51.055858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:51.055858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:51.055858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:51.055858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:51.055858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:51.055858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:51.055858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:55:51.056923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:55:51.056923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:51.056923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:51.060693 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 195/397] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:51.061404 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c0128-2b23-4c34-99ff-6fe26ab6d7d5 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:51.073831 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:51.130996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:51.154140 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463d9ff-bf82-40e4-8d74-13ce1755dd2c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:51.166164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c0128-2b23-4c34-99ff-6fe26ab6d7d5 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:51.187369 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f8d71-931f-4474-8e02-eccf6d98ec13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:51.187967 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f8d71-931f-4474-8e02-eccf6d98ec13 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:51.188798 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f8d71-931f-4474-8e02-eccf6d98ec13 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:51.189082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f8d71-931f-4474-8e02-eccf6d98ec13 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:51.189716 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f8d71-931f-4474-8e02-eccf6d98ec13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:51.190023 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f8d71-931f-4474-8e02-eccf6d98ec13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:51.191382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f8d71-931f-4474-8e02-eccf6d98ec13 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:51.200389 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09d5b6-9567-4b24-be5c-11e460988911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:51.200666 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09d5b6-9567-4b24-be5c-11e460988911 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:51.200947 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09d5b6-9567-4b24-be5c-11e460988911 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:51.201227 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09d5b6-9567-4b24-be5c-11e460988911 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:51.201859 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09d5b6-9567-4b24-be5c-11e460988911 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:51.202158 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09d5b6-9567-4b24-be5c-11e460988911 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:51.203284 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 203/398] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:51.203510 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba09d5b6-9567-4b24-be5c-11e460988911 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:51.204752 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f8d71-931f-4474-8e02-eccf6d98ec13 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:51.215026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 196/399] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:51.216887 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09d5b6-9567-4b24-be5c-11e460988911 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:51.286795 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:51.305508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f8d71-931f-4474-8e02-eccf6d98ec13 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:51.314984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09d5b6-9567-4b24-be5c-11e460988911 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:51.339278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755a723-aa56-4c14-b2ea-999396b5082b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:51.339911 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755a723-aa56-4c14-b2ea-999396b5082b None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:51.340192 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755a723-aa56-4c14-b2ea-999396b5082b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:51.340470 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755a723-aa56-4c14-b2ea-999396b5082b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:51.341636 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:51.351140 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755a723-aa56-4c14-b2ea-999396b5082b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:51.351296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2351e-b885-42d0-9edc-7ec06c2ce721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:51.351852 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2351e-b885-42d0-9edc-7ec06c2ce721 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:51.352135 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2351e-b885-42d0-9edc-7ec06c2ce721 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:51.352407 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2351e-b885-42d0-9edc-7ec06c2ce721 None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:51.352607 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755a723-aa56-4c14-b2ea-999396b5082b None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:51.353159 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2351e-b885-42d0-9edc-7ec06c2ce721 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:51.353425 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2351e-b885-42d0-9edc-7ec06c2ce721 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:51.354824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2351e-b885-42d0-9edc-7ec06c2ce721 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:51.367279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-c755a723-aa56-4c14-b2ea-999396b5082b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:51.367279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:51.367279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:51.367279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:51.367279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:51.367279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:51.367279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:51.367279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:51.367279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:51.367279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:51.367279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:51.367279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:51.367279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:55:51.367279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:55:51.367279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:55:51.367279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:55:51.368193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:51.368193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:51.368193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:51.368193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:51.368193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:51.368193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:51.368193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:51.368193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:51.368193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:51.368193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:51.368193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:51.368193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:51.368193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:51.368193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:51.368193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:51.368193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:51.368193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:55:51.369942 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:55:51.369942 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:51.369942 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:51.369942 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 204/400] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:51.369942 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755a723-aa56-4c14-b2ea-999396b5082b None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:51.373733 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 197/401] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Oct 22 23:55:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:51.374646 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2351e-b885-42d0-9edc-7ec06c2ce721 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:51.472507 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755a723-aa56-4c14-b2ea-999396b5082b None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:51.473566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2351e-b885-42d0-9edc-7ec06c2ce721 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:51.506658 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9dba1-b551-424d-964f-9fe8731e8cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:51.507297 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9dba1-b551-424d-964f-9fe8731e8cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:51.507528 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0727f-2e9a-46fe-8dcc-eed4d21381b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:51.507662 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9dba1-b551-424d-964f-9fe8731e8cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:51.507888 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9dba1-b551-424d-964f-9fe8731e8cf5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:51.508190 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0727f-2e9a-46fe-8dcc-eed4d21381b6 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:51.508480 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0727f-2e9a-46fe-8dcc-eed4d21381b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:51.508783 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0727f-2e9a-46fe-8dcc-eed4d21381b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:51.509422 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0727f-2e9a-46fe-8dcc-eed4d21381b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:51.509728 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0727f-2e9a-46fe-8dcc-eed4d21381b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:51.511085 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0727f-2e9a-46fe-8dcc-eed4d21381b6 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:51.512532 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:51.514640 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9dba1-b551-424d-964f-9fe8731e8cf5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:51.516085 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9dba1-b551-424d-964f-9fe8731e8cf5 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:51.524530 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 205/402] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:51.525344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0727f-2e9a-46fe-8dcc-eed4d21381b6 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:51.529108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-edf9dba1-b551-424d-964f-9fe8731e8cf5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:55:51.529108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:51.529108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:51.529108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:51.529108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:51.529108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:51.529108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:51.529108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:51.529108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:51.529108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:51.529108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:51.529108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:51.529108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:55:51.529108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:55:51.529108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:55:51.530127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:55:51.530127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:51.530127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:51.530127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:51.530127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:51.530127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:51.530127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:51.530127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:51.530127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:51.530127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:51.530127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:51.530127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:51.530127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:51.530127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:51.530127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:51.530127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:51.530127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:51.530979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:55:51.530979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:55:51.530979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:55:51.530979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:55:51.530979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:55:51.530979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:55:51.530979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:55:51.530979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:51.530979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 198/403] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:55:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:51.530979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9dba1-b551-424d-964f-9fe8731e8cf5 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:51.581305 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:51.627918 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0727f-2e9a-46fe-8dcc-eed4d21381b6 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:51.630632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9dba1-b551-424d-964f-9fe8731e8cf5 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:51.661079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1bcfb-9eba-4983-aadf-dcd1f8421042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:51.661698 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1bcfb-9eba-4983-aadf-dcd1f8421042 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:51.661984 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1bcfb-9eba-4983-aadf-dcd1f8421042 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:51.662260 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1bcfb-9eba-4983-aadf-dcd1f8421042 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:51.663791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80cb01-35e9-4530-8a7e-95303e11e939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:51.664384 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80cb01-35e9-4530-8a7e-95303e11e939 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:51.664704 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80cb01-35e9-4530-8a7e-95303e11e939 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:51.664978 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80cb01-35e9-4530-8a7e-95303e11e939 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:51.665608 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80cb01-35e9-4530-8a7e-95303e11e939 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:51.665905 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80cb01-35e9-4530-8a7e-95303e11e939 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:51.667270 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80cb01-35e9-4530-8a7e-95303e11e939 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:51.668852 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1bcfb-9eba-4983-aadf-dcd1f8421042 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:51.670282 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f1bcfb-9eba-4983-aadf-dcd1f8421042 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:51.678686 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 199/404] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:51.679503 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80cb01-35e9-4530-8a7e-95303e11e939 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:51.681827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-62f1bcfb-9eba-4983-aadf-dcd1f8421042 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:55:51.681827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:51.681827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:51.681827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:51.681827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:51.681827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:51.681827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:51.681827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:51.681827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:51.681827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:51.681827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:51.681827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:51.681827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:55:51.681827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:55:51.681827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:55:51.682663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:55:51.682663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:51.682663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:51.682663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:51.682663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:51.682663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:51.682663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:51.682663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:51.682663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:51.682663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:51.682663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:51.682663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:51.682663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:51.682663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:51.682663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:51.682663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:51.682663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:51.683485 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:55:51.683485 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:55:51.683485 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:55:51.683485 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:55:51.683485 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:55:51.683485 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:55:51.683485 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:55:51.683485 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:51.683485 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 206/405] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:55:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:51.683960 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1bcfb-9eba-4983-aadf-dcd1f8421042 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:51.738974 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:51.781035 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80cb01-35e9-4530-8a7e-95303e11e939 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:51.790585 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1bcfb-9eba-4983-aadf-dcd1f8421042 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:51.791716 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:51.816783 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89b996-1708-4b4d-827a-0663c0a0a9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:51.817693 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89b996-1708-4b4d-827a-0663c0a0a9fc None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:51.817977 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89b996-1708-4b4d-827a-0663c0a0a9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:51.818255 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89b996-1708-4b4d-827a-0663c0a0a9fc None admin] PATH_INFO: `/v3/projects/5c9246c059754504afee4711d113a3fe/users/c4a3579d56294b9d8ad36d4d174c8526/roles/73721a054c3842bba868ab52c1a3f8e0` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:51.824452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aaabaa-80e5-45ca-a198-022b1d6218ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:51.825106 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aaabaa-80e5-45ca-a198-022b1d6218ff None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:51.825389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aaabaa-80e5-45ca-a198-022b1d6218ff None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:51.825662 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aaabaa-80e5-45ca-a198-022b1d6218ff None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:51.833031 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd89b996-1708-4b4d-827a-0663c0a0a9fc None admin] RBAC: Authorizing `identity:create_grant(project_id=5c9246c059754504afee4711d113a3fe, user_id=c4a3579d56294b9d8ad36d4d174c8526, role_id=73721a054c3842bba868ab52c1a3f8e0)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:51.833753 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aaabaa-80e5-45ca-a198-022b1d6218ff None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:51.834427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd89b996-1708-4b4d-827a-0663c0a0a9fc None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:51.835163 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aaabaa-80e5-45ca-a198-022b1d6218ff None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:51.846393 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-d4aaabaa-80e5-45ca-a198-022b1d6218ff None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 22 23:55:51.846393 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:51.846393 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:51.846393 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:51.846393 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:51.846393 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:51.846393 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:51.846393 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:51.846393 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:51.846393 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:51.846393 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:51.846393 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:51.846393 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:55:51.846393 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:55:51.846393 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:55:51.846393 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:55:51.847501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:51.847501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:51.847501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:55:51.847501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:55:51.847501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:55:51.847501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:55:51.847501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:51.847501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:51.847501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:51.847501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:51.847501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:51.847501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:51.847501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:51.847501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:51.847501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:51.847501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:51.847501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:51.847501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:51.848845 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:51.848845 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:51.848845 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:55:51.848845 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:55:51.848845 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:55:51.848845 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:55:51.848845 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:55:51.848845 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:55:51.848845 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 22 23:55:51.848845 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:51.849482 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 200/406] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:55:51 2022] PUT /identity/v3/projects/5c9246c059754504afee4711d113a3fe/users/c4a3579d56294b9d8ad36d4d174c8526/roles/73721a054c3842bba868ab52c1a3f8e0 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:55:51.850031 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89b996-1708-4b4d-827a-0663c0a0a9fc None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:51.852372 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 207/407] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Oct 22 23:55:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:51.852986 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aaabaa-80e5-45ca-a198-022b1d6218ff None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:51.966435 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89b996-1708-4b4d-827a-0663c0a0a9fc None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:51.980828 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aaabaa-80e5-45ca-a198-022b1d6218ff None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:51.998470 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:52.005048 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68277d-fbaf-4e66-86bd-c8d32822c0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:52.005765 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68277d-fbaf-4e66-86bd-c8d32822c0ee None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:52.006110 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68277d-fbaf-4e66-86bd-c8d32822c0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:52.006457 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68277d-fbaf-4e66-86bd-c8d32822c0ee None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:52.007214 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68277d-fbaf-4e66-86bd-c8d32822c0ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:52.008013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68277d-fbaf-4e66-86bd-c8d32822c0ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:52.009126 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68277d-fbaf-4e66-86bd-c8d32822c0ee None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:52.022757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 201/408] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Oct 22 23:55:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:52.027623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68277d-fbaf-4e66-86bd-c8d32822c0ee None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:52.027747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f01fc0-bcf7-4f77-811c-708a457aebf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:52.027747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f01fc0-bcf7-4f77-811c-708a457aebf2 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:52.027747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f01fc0-bcf7-4f77-811c-708a457aebf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:52.027747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f01fc0-bcf7-4f77-811c-708a457aebf2 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:52.028077 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f01fc0-bcf7-4f77-811c-708a457aebf2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:52.028890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f01fc0-bcf7-4f77-811c-708a457aebf2 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:52.049962 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:52.056541 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 208/409] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Oct 22 23:55:51 2022] POST /identity/v3/users => generated 247 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:55:52.056541 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f01fc0-bcf7-4f77-811c-708a457aebf2 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:52.179663 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68277d-fbaf-4e66-86bd-c8d32822c0ee None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:52.187010 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f01fc0-bcf7-4f77-811c-708a457aebf2 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:52.219387 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f04b79-4220-4f94-8fd6-4a904c49772e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:52.220140 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f04b79-4220-4f94-8fd6-4a904c49772e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:52.220424 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f04b79-4220-4f94-8fd6-4a904c49772e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:52.220755 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f04b79-4220-4f94-8fd6-4a904c49772e None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:52.221506 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f04b79-4220-4f94-8fd6-4a904c49772e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:52.221852 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f04b79-4220-4f94-8fd6-4a904c49772e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:52.224154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f04b79-4220-4f94-8fd6-4a904c49772e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:52.233345 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc49e89-476c-45a2-a384-315003f021c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:52.234603 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc49e89-476c-45a2-a384-315003f021c5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:52.235681 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc49e89-476c-45a2-a384-315003f021c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:52.236228 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc49e89-476c-45a2-a384-315003f021c5 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:52.237183 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc49e89-476c-45a2-a384-315003f021c5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:52.237672 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc49e89-476c-45a2-a384-315003f021c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:52.243228 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 202/410] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Oct 22 23:55:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:52.243228 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f04b79-4220-4f94-8fd6-4a904c49772e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:52.243457 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc49e89-476c-45a2-a384-315003f021c5 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:52.263922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 209/411] 10.222.0.13 () {56 vars in 1139 bytes} [Sat Oct 22 23:55:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:52.265420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc49e89-476c-45a2-a384-315003f021c5 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:52.301658 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:52.351135 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:52.375447 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f04b79-4220-4f94-8fd6-4a904c49772e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:52.402269 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc49e89-476c-45a2-a384-315003f021c5 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:52.411809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5139885-9b09-4a1e-969e-599c8920dd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:52.412619 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5139885-9b09-4a1e-969e-599c8920dd42 None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:52.412965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5139885-9b09-4a1e-969e-599c8920dd42 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:52.413309 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5139885-9b09-4a1e-969e-599c8920dd42 None admin] PATH_INFO: `/v3/projects/5c9246c059754504afee4711d113a3fe/users/bd170de478a344fc98951697c7a6298d/roles/73721a054c3842bba868ab52c1a3f8e0` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:52.429096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5139885-9b09-4a1e-969e-599c8920dd42 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c9246c059754504afee4711d113a3fe, user_id=bd170de478a344fc98951697c7a6298d, role_id=73721a054c3842bba868ab52c1a3f8e0)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:52.430652 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5139885-9b09-4a1e-969e-599c8920dd42 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:52.449320 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 203/412] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:55:52 2022] PUT /identity/v3/projects/5c9246c059754504afee4711d113a3fe/users/bd170de478a344fc98951697c7a6298d/roles/73721a054c3842bba868ab52c1a3f8e0 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:55:52.450353 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5139885-9b09-4a1e-969e-599c8920dd42 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:52.455559 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7a09c-72f1-4d2d-b0af-310fd0b867f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:52.461195 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7a09c-72f1-4d2d-b0af-310fd0b867f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:52.461484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7a09c-72f1-4d2d-b0af-310fd0b867f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:52.461884 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7a09c-72f1-4d2d-b0af-310fd0b867f9 None admin] PATH_INFO: `/v3/projects/5c9246c059754504afee4711d113a3fe/users/e4a69aeb3fba49b6881f1f15c1758494/roles/73721a054c3842bba868ab52c1a3f8e0` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:52.490085 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7a09c-72f1-4d2d-b0af-310fd0b867f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c9246c059754504afee4711d113a3fe, user_id=e4a69aeb3fba49b6881f1f15c1758494, role_id=73721a054c3842bba868ab52c1a3f8e0)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:52.491614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7a09c-72f1-4d2d-b0af-310fd0b867f9 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:52.520704 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 210/413] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:55:52 2022] PUT /identity/v3/projects/5c9246c059754504afee4711d113a3fe/users/e4a69aeb3fba49b6881f1f15c1758494/roles/73721a054c3842bba868ab52c1a3f8e0 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:55:52.616865 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5139885-9b09-4a1e-969e-599c8920dd42 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:52.661647 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:52.674819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4646a-e6d9-4aff-ab57-2c84a1706efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:52.675488 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4646a-e6d9-4aff-ab57-2c84a1706efc None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:52.675783 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4646a-e6d9-4aff-ab57-2c84a1706efc None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:52.676062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4646a-e6d9-4aff-ab57-2c84a1706efc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:52.694230 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4646a-e6d9-4aff-ab57-2c84a1706efc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:52.695720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4646a-e6d9-4aff-ab57-2c84a1706efc None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:52.713103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-7ec4646a-e6d9-4aff-ab57-2c84a1706efc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:52.713103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:52.713103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:52.713103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:52.713103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:52.713103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:52.713103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:52.713103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:52.713103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:52.713103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:52.713103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:52.713103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:52.713103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:55:52.713103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:55:52.713103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:55:52.713103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:55:52.714086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:52.714086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:52.714086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:52.714086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:52.714086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:52.714086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:52.714086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:52.714086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:52.714086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:52.714086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:52.714086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:52.714086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:52.714086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:52.714086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:52.714086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:52.714086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:52.714086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:55:52.714947 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:55:52.714947 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:52.714947 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:52.714947 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 204/414] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:52.720855 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7a09c-72f1-4d2d-b0af-310fd0b867f9 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:52.831127 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:52.859012 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7a09c-72f1-4d2d-b0af-310fd0b867f9 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:52.897791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d6688-9ae3-4310-938c-fff25e173500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:52.898381 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d6688-9ae3-4310-938c-fff25e173500 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:52.898660 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d6688-9ae3-4310-938c-fff25e173500 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:52.898933 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d6688-9ae3-4310-938c-fff25e173500 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:52.899576 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d6688-9ae3-4310-938c-fff25e173500 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:52.899875 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d6688-9ae3-4310-938c-fff25e173500 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:52.901256 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0d6688-9ae3-4310-938c-fff25e173500 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:52.921759 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 211/415] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:55:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:52.927325 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4646a-e6d9-4aff-ab57-2c84a1706efc None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:53.073811 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4646a-e6d9-4aff-ab57-2c84a1706efc None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:53.120616 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbce087-49ba-4a10-893e-98090dd384a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:53.121254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbce087-49ba-4a10-893e-98090dd384a4 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:53.121535 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbce087-49ba-4a10-893e-98090dd384a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:53.121808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbce087-49ba-4a10-893e-98090dd384a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:53.135134 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbce087-49ba-4a10-893e-98090dd384a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:53.136605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbce087-49ba-4a10-893e-98090dd384a4 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:53.153557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-5dbce087-49ba-4a10-893e-98090dd384a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:55:53.153557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:53.153557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:53.153557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:53.153557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:53.153557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:53.153557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:53.153557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:53.153557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:53.153557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:53.153557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:53.153557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:53.153557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:55:53.153557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:55:53.153557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:55:53.154537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:55:53.154537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:53.154537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:53.154537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:53.154537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:53.154537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:53.154537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:53.154537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:53.154537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:53.154537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:53.154537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:53.154537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:53.154537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:53.154537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:53.154537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:53.154537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:53.154537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:53.155398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:55:53.155398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:55:53.155398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:55:53.155398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:55:53.155398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:55:53.155398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:55:53.155398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:55:53.155398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:53.159305 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 205/416] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:55:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:53.160652 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d6688-9ae3-4310-938c-fff25e173500 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:53.309453 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0d6688-9ae3-4310-938c-fff25e173500 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:53.345720 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ce40b-6f12-4d8c-9a83-54c6983d0112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:53.346332 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ce40b-6f12-4d8c-9a83-54c6983d0112 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:53.346620 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ce40b-6f12-4d8c-9a83-54c6983d0112 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:53.346900 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ce40b-6f12-4d8c-9a83-54c6983d0112 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:53.347593 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ce40b-6f12-4d8c-9a83-54c6983d0112 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:53.347905 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ce40b-6f12-4d8c-9a83-54c6983d0112 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:53.349357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ce40b-6f12-4d8c-9a83-54c6983d0112 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:53.360194 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:53.368679 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 212/417] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Oct 22 23:55:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:53.374160 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbce087-49ba-4a10-893e-98090dd384a4 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:53.446377 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:53.511618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbce087-49ba-4a10-893e-98090dd384a4 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:53.569272 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f625432-172b-4fc0-8b10-bb793e3661e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:53.569864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f625432-172b-4fc0-8b10-bb793e3661e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:53.570147 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f625432-172b-4fc0-8b10-bb793e3661e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:53.570425 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f625432-172b-4fc0-8b10-bb793e3661e5 None admin] PATH_INFO: `/v3/projects/5c9246c059754504afee4711d113a3fe/users/a09d45d080bc461c88f38ff281b75b06/roles/73721a054c3842bba868ab52c1a3f8e0` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:53.586245 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f625432-172b-4fc0-8b10-bb793e3661e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c9246c059754504afee4711d113a3fe, user_id=a09d45d080bc461c88f38ff281b75b06, role_id=73721a054c3842bba868ab52c1a3f8e0)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:53.587673 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f625432-172b-4fc0-8b10-bb793e3661e5 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:53.610276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 206/418] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:55:53 2022] PUT /identity/v3/projects/5c9246c059754504afee4711d113a3fe/users/a09d45d080bc461c88f38ff281b75b06/roles/73721a054c3842bba868ab52c1a3f8e0 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:55:54.338702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e8414-a35f-43c1-a68b-aa81f94262d4 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:54.339210 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e8414-a35f-43c1-a68b-aa81f94262d4 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:54.339285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e8414-a35f-43c1-a68b-aa81f94262d4 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:54.340904 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 213/419] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:54.346757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4e64b-4c68-417c-80ad-0644545500ab None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:54.347046 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4e64b-4c68-417c-80ad-0644545500ab None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:54.347321 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4e64b-4c68-417c-80ad-0644545500ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:54.373003 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-bec4e64b-4c68-417c-80ad-0644545500ab None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:54.374132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-bec4e64b-4c68-417c-80ad-0644545500ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:54.452726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 207/420] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:54.723942 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c339f8b-b761-4b5a-ad14-90738bcdc736 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:54.724126 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c339f8b-b761-4b5a-ad14-90738bcdc736 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:54.724403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c339f8b-b761-4b5a-ad14-90738bcdc736 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:54.725896 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 214/421] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:54.730923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87866691-590c-455f-adf1-08e15931eceb None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:54.731215 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87866691-590c-455f-adf1-08e15931eceb None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:54.731494 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87866691-590c-455f-adf1-08e15931eceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:54.754556 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-87866691-590c-455f-adf1-08e15931eceb None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:54.755736 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-87866691-590c-455f-adf1-08e15931eceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:54.771392 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:54.776789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c339f8b-b761-4b5a-ad14-90738bcdc736 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:54.861427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 208/422] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:54.919403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c339f8b-b761-4b5a-ad14-90738bcdc736 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:54.966122 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd3ccf-97d5-412a-a650-3a9dc162bccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:54.966758 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd3ccf-97d5-412a-a650-3a9dc162bccc None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:54.967042 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd3ccf-97d5-412a-a650-3a9dc162bccc None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:54.967318 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd3ccf-97d5-412a-a650-3a9dc162bccc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:54.985866 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd3ccf-97d5-412a-a650-3a9dc162bccc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:54.990668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-5ebd3ccf-97d5-412a-a650-3a9dc162bccc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:55:54.990668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:54.990668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:54.990668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:54.990668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:54.990668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:54.990668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:54.990668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:54.990668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:54.990668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:54.990668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:54.990668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:54.990668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:54.990668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:54.990668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:54.990668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:54.991564 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:54.991564 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:54.991564 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:54.991564 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:54.991564 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:54.991564 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:54.991564 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:54.991564 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:54.991564 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:54.991564 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:54.991564 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:54.991564 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:54.991564 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:54.991564 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:54.991564 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:54.991564 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:54.992382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:54.992382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:54.992382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:54.992382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:54.992382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:54.992382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:54.992382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:54.992382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:54.992382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:54.992382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:54.992382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:55:54.992382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:54.992382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 215/423] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Oct 22 23:55:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:54.997820 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87866691-590c-455f-adf1-08e15931eceb None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:55.137574 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd3ccf-97d5-412a-a650-3a9dc162bccc None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:55.156936 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87866691-590c-455f-adf1-08e15931eceb None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:55.194467 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb528e8-0487-4c80-83e8-b26a005d3532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:55.195073 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb528e8-0487-4c80-83e8-b26a005d3532 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:55.195350 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb528e8-0487-4c80-83e8-b26a005d3532 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:55.195619 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb528e8-0487-4c80-83e8-b26a005d3532 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:55.196279 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb528e8-0487-4c80-83e8-b26a005d3532 None admin] RBAC: Adding query filter params (name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:55.196607 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb528e8-0487-4c80-83e8-b26a005d3532 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:55.197976 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb528e8-0487-4c80-83e8-b26a005d3532 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:55.207314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 209/424] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Oct 22 23:55:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:55.223457 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb528e8-0487-4c80-83e8-b26a005d3532 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:55.244431 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:55.254500 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:55.315586 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd3ccf-97d5-412a-a650-3a9dc162bccc None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:55.356685 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb528e8-0487-4c80-83e8-b26a005d3532 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:55.362047 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee986b-631f-440a-a651-fce9dce0e562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:55.362792 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee986b-631f-440a-a651-fce9dce0e562 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:55.363146 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee986b-631f-440a-a651-fce9dce0e562 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:55.363489 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee986b-631f-440a-a651-fce9dce0e562 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:55.377198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee986b-631f-440a-a651-fce9dce0e562 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:55.383207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-a4ee986b-631f-440a-a651-fce9dce0e562 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:55:55.383207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:55.383207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:55.383207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:55.383207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:55.383207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:55.383207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:55.383207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:55.383207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:55.383207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:55.383207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:55.383207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:55.383207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:55.383207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:55.383207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:55.383207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:55.384091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:55.384091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:55.384091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:55.384091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:55.384091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:55.384091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:55.384091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:55.384091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:55.384091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:55.384091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:55.384091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:55.384091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:55.384091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:55.384091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:55.384091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:55.384091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:55.384949 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:55.384949 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:55.384949 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:55.384949 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:55.384949 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:55.384949 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:55.384949 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:55.384949 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:55.384949 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:55.384949 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:55.384949 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:55:55.384949 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:55.384949 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 216/425] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Oct 22 23:55:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:55.388509 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38dbd0-f33a-41f3-b5ee-ecdfa3e8ab1b None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:55.388509 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38dbd0-f33a-41f3-b5ee-ecdfa3e8ab1b None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:55.388509 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38dbd0-f33a-41f3-b5ee-ecdfa3e8ab1b None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:55.389419 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 217/426] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:55.390137 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38dbd0-f33a-41f3-b5ee-ecdfa3e8ab1b None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:55.398598 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c84d0-1628-4b41-9d9b-92c9c5d9a8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:55.399396 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c84d0-1628-4b41-9d9b-92c9c5d9a8be None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:55.399773 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c84d0-1628-4b41-9d9b-92c9c5d9a8be None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:55.400140 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c84d0-1628-4b41-9d9b-92c9c5d9a8be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:55.411585 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c84d0-1628-4b41-9d9b-92c9c5d9a8be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:55.413235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c84d0-1628-4b41-9d9b-92c9c5d9a8be None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:55.428605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-d18c84d0-1628-4b41-9d9b-92c9c5d9a8be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:55.428605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:55.428605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:55.428605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:55.428605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:55.428605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:55.428605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:55.428605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:55.428605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:55.428605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:55.428605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:55.428605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:55.428605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:55:55.428605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:55:55.428605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:55:55.428605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:55:55.429806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:55.429806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:55.429806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:55.429806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:55.429806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:55.429806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:55.429806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:55.429806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:55.429806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:55.429806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:55.429806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:55.429806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:55.429806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:55.429806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:55.429806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:55.429806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:55.429806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:55:55.430890 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:55:55.430890 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:55.430890 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:55.436135 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 210/427] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:55:55.436135 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4fa720-6d90-4e20-baa9-ac766298d722 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:55.436135 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4fa720-6d90-4e20-baa9-ac766298d722 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:55.436135 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4fa720-6d90-4e20-baa9-ac766298d722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:55.455932 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-0f4fa720-6d90-4e20-baa9-ac766298d722 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:55.457126 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-0f4fa720-6d90-4e20-baa9-ac766298d722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:55.510993 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38dbd0-f33a-41f3-b5ee-ecdfa3e8ab1b None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:55.522063 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:55.529917 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:55.533952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 211/428] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:55.534991 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4fa720-6d90-4e20-baa9-ac766298d722 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:55.561261 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16007561-e2ae-43f6-a178-d5e0a360877c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:55.561878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16007561-e2ae-43f6-a178-d5e0a360877c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:55.562207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16007561-e2ae-43f6-a178-d5e0a360877c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:55.562436 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16007561-e2ae-43f6-a178-d5e0a360877c None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:55.563101 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16007561-e2ae-43f6-a178-d5e0a360877c None admin] RBAC: Adding query filter params (name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:55.563991 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16007561-e2ae-43f6-a178-d5e0a360877c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:55.565391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16007561-e2ae-43f6-a178-d5e0a360877c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:55.578315 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 218/429] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Oct 22 23:55:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:55.578315 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acdf90-bb53-4bca-b471-042900f048ed None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:55.578315 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acdf90-bb53-4bca-b471-042900f048ed None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:55.578315 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acdf90-bb53-4bca-b471-042900f048ed None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:55.578315 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 219/430] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:55.579276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22acdf90-bb53-4bca-b471-042900f048ed None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:55.681740 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4fa720-6d90-4e20-baa9-ac766298d722 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:55.695237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22acdf90-bb53-4bca-b471-042900f048ed None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:55.720134 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543955fa-169e-4b83-b97e-3cac414928bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:55.721164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543955fa-169e-4b83-b97e-3cac414928bb None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:55.721754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543955fa-169e-4b83-b97e-3cac414928bb None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:55.722212 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543955fa-169e-4b83-b97e-3cac414928bb None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:55.723086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543955fa-169e-4b83-b97e-3cac414928bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:55.723580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543955fa-169e-4b83-b97e-3cac414928bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:55.725437 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543955fa-169e-4b83-b97e-3cac414928bb None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:55.733473 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0663cf5-c7b3-4e29-9649-f82f7bb5c959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:55.734190 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0663cf5-c7b3-4e29-9649-f82f7bb5c959 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:55.734533 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0663cf5-c7b3-4e29-9649-f82f7bb5c959 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:55.734873 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0663cf5-c7b3-4e29-9649-f82f7bb5c959 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:55.743369 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 212/431] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:55.745047 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812846a-c42c-4915-b8a9-524c04cc0ec6 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:55.745521 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812846a-c42c-4915-b8a9-524c04cc0ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:55.745979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812846a-c42c-4915-b8a9-524c04cc0ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:55.746677 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0663cf5-c7b3-4e29-9649-f82f7bb5c959 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:55.748184 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0663cf5-c7b3-4e29-9649-f82f7bb5c959 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:55.763813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-b0663cf5-c7b3-4e29-9649-f82f7bb5c959 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:55.763813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:55.763813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:55.763813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:55.763813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:55.763813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:55.763813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:55.763813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:55.763813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:55.763813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:55.763813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:55.763813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:55.763813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:55:55.763813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:55:55.763813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:55:55.763813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:55:55.764767 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:55.764767 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:55.764767 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:55.764767 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:55.764767 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:55.764767 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:55.764767 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:55.764767 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:55.764767 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:55.764767 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:55.764767 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:55.764767 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:55.764767 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:55.764767 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:55.764767 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:55.764767 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:55.764767 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:55:55.765705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:55:55.765705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:55.765705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:55.768957 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-d812846a-c42c-4915-b8a9-524c04cc0ec6 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:55.769816 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 220/432] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:55.770594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d812846a-c42c-4915-b8a9-524c04cc0ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:55.770719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0663cf5-c7b3-4e29-9649-f82f7bb5c959 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:55.806714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:55.831146 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:55.860831 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 213/433] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:55.860831 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812846a-c42c-4915-b8a9-524c04cc0ec6 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:55.898935 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0663cf5-c7b3-4e29-9649-f82f7bb5c959 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:55.939868 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4525b-a067-4247-85d0-50cec13cac9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:55.940796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4525b-a067-4247-85d0-50cec13cac9c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:55.941081 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4525b-a067-4247-85d0-50cec13cac9c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:55.941358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4525b-a067-4247-85d0-50cec13cac9c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:55.951423 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4525b-a067-4247-85d0-50cec13cac9c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:55.953029 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4525b-a067-4247-85d0-50cec13cac9c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:55.964513 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812846a-c42c-4915-b8a9-524c04cc0ec6 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:55.969357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-5fd4525b-a067-4247-85d0-50cec13cac9c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 22 23:55:55.969357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:55.969357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:55.969357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:55.969357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:55.969357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:55.969357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:55.969357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:55.969357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:55.969357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:55.969357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:55.969357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:55.969357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:55:55.969357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:55:55.969357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:55:55.969357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:55:55.970355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:55.970355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:55.970355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:55:55.970355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:55:55.970355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:55:55.970355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:55:55.970355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:55.970355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:55.970355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:55.970355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:55.970355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:55.970355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:55.970355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:55.970355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:55.970355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:55.970355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:55.970355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:55.970355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:55.971818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:55.971818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:55.971818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:55:55.971818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:55:55.971818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:55:55.971818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:55:55.971818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:55:55.971818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:55:55.971818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 22 23:55:55.971818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:55.971818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 221/434] 10.222.0.13 () {56 vars in 1093 bytes} [Sat Oct 22 23:55:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:55.971818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4525b-a067-4247-85d0-50cec13cac9c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:56.009559 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328b916-68c1-42dc-9c2a-8aa33108ccee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:56.010175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328b916-68c1-42dc-9c2a-8aa33108ccee None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:56.010462 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328b916-68c1-42dc-9c2a-8aa33108ccee None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:56.010741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328b916-68c1-42dc-9c2a-8aa33108ccee None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:56.011384 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328b916-68c1-42dc-9c2a-8aa33108ccee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:56.011692 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328b916-68c1-42dc-9c2a-8aa33108ccee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:56.014446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328b916-68c1-42dc-9c2a-8aa33108ccee None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:56.034622 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 214/435] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:56.034622 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328b916-68c1-42dc-9c2a-8aa33108ccee None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:56.085129 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:56.096690 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4525b-a067-4247-85d0-50cec13cac9c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:56.096842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:56.138130 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218c177-a988-4124-90f9-edb14a66e5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:56.139207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218c177-a988-4124-90f9-edb14a66e5c3 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:56.139806 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218c177-a988-4124-90f9-edb14a66e5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:56.140273 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218c177-a988-4124-90f9-edb14a66e5c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:56.152646 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f218c177-a988-4124-90f9-edb14a66e5c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:56.159529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-f218c177-a988-4124-90f9-edb14a66e5c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:55:56.159529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:56.159529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:56.159529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:56.159529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:56.159529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:56.159529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:56.159529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:56.159529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:56.159529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:56.159529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:56.159529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:56.159529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:56.159529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:56.159529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:56.159529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:56.160408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:56.160408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:56.160408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:56.160408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:56.160408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:56.160408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:56.160408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:56.160408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:56.160408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:56.160408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:56.160408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:56.160408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:56.160408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:56.160408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:56.160408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:56.160408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:56.161214 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:56.161214 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:56.161214 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:56.161214 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:56.161214 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:56.161214 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:56.161214 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:56.161214 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:56.161214 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:56.161214 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:56.161214 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:55:56.161214 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:56.163883 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 222/436] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Oct 22 23:55:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:56.165428 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218c177-a988-4124-90f9-edb14a66e5c3 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:56.178756 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328b916-68c1-42dc-9c2a-8aa33108ccee None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:56.223629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa8a2b-6c9b-4db1-94a0-f4dd51089408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:56.224246 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa8a2b-6c9b-4db1-94a0-f4dd51089408 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:56.224555 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa8a2b-6c9b-4db1-94a0-f4dd51089408 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:56.224835 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa8a2b-6c9b-4db1-94a0-f4dd51089408 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:56.225564 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa8a2b-6c9b-4db1-94a0-f4dd51089408 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:56.225871 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa8a2b-6c9b-4db1-94a0-f4dd51089408 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:56.227332 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa8a2b-6c9b-4db1-94a0-f4dd51089408 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:56.246001 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 215/437] 10.222.0.13 () {56 vars in 1133 bytes} [Sat Oct 22 23:55:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:56.246654 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa8a2b-6c9b-4db1-94a0-f4dd51089408 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:56.277625 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218c177-a988-4124-90f9-edb14a66e5c3 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:56.314635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f9399-981c-4ade-8100-267f09b61e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:56.315580 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f9399-981c-4ade-8100-267f09b61e32 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:56.316091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f9399-981c-4ade-8100-267f09b61e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:56.316560 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f9399-981c-4ade-8100-267f09b61e32 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:56.327343 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f9399-981c-4ade-8100-267f09b61e32 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:56.330183 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f9399-981c-4ade-8100-267f09b61e32 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:56.342764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-815f9399-981c-4ade-8100-267f09b61e32 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 22 23:55:56.342764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:56.342764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:56.342764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:56.342764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:56.342764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:56.342764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:56.342764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:56.342764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:56.342764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:56.342764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:56.342764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:56.342764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:55:56.342764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:55:56.342764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:55:56.342764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:55:56.343724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:56.343724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:56.343724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:55:56.343724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:55:56.343724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:55:56.343724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:55:56.343724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:56.343724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:56.343724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:56.343724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:56.343724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:56.343724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:56.343724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:56.343724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:56.343724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:56.343724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:56.343724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:56.343724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:56.350803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:56.350803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:56.350803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:55:56.350803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:55:56.350803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:55:56.350803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:55:56.350803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:55:56.350803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:55:56.350803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 22 23:55:56.350803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:56.350803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 223/438] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Oct 22 23:55:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:56.350803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f9399-981c-4ade-8100-267f09b61e32 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:56.360199 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa8a2b-6c9b-4db1-94a0-f4dd51089408 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:56.361960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:56.366003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:56.398464 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc13e57-6ce3-4d18-b626-cb7b28f0cd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:56.399109 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc13e57-6ce3-4d18-b626-cb7b28f0cd4e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:56.399384 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc13e57-6ce3-4d18-b626-cb7b28f0cd4e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:56.399661 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc13e57-6ce3-4d18-b626-cb7b28f0cd4e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:56.411718 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc13e57-6ce3-4d18-b626-cb7b28f0cd4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:56.416576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-1cc13e57-6ce3-4d18-b626-cb7b28f0cd4e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:55:56.416576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:56.416576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:56.416576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:56.416576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:56.416576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:56.416576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:56.416576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:56.416576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:56.416576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:56.416576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:56.416576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:56.416576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:56.416576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:56.416576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:56.416576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:56.417499 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:56.417499 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:56.417499 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:56.417499 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:56.417499 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:56.417499 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:56.417499 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:56.417499 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:56.417499 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:56.417499 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:56.417499 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:56.417499 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:56.417499 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:56.417499 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:56.417499 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:56.417499 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:56.418296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:56.418296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:56.418296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:56.418296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:56.418296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:56.418296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:56.418296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:56.418296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:56.418296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:56.418296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:56.418296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:55:56.418296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:56.418296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 216/439] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Oct 22 23:55:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:56.419043 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc13e57-6ce3-4d18-b626-cb7b28f0cd4e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:56.469220 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f9399-981c-4ade-8100-267f09b61e32 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:56.502989 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a72680-7d3d-44b2-bde1-ec2d7b68fa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:56.503585 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a72680-7d3d-44b2-bde1-ec2d7b68fa64 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:56.503870 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a72680-7d3d-44b2-bde1-ec2d7b68fa64 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:56.504148 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a72680-7d3d-44b2-bde1-ec2d7b68fa64 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:56.504838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a72680-7d3d-44b2-bde1-ec2d7b68fa64 None admin] RBAC: Adding query filter params (name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:56.505145 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a72680-7d3d-44b2-bde1-ec2d7b68fa64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:56.506502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a72680-7d3d-44b2-bde1-ec2d7b68fa64 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:56.514142 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 224/440] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Oct 22 23:55:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:56.514924 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a72680-7d3d-44b2-bde1-ec2d7b68fa64 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:56.521465 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc13e57-6ce3-4d18-b626-cb7b28f0cd4e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:56.556510 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07811158-6464-40da-b409-b3fc7494cdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:56.557042 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07811158-6464-40da-b409-b3fc7494cdfb None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:56.557324 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07811158-6464-40da-b409-b3fc7494cdfb None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:56.557600 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07811158-6464-40da-b409-b3fc7494cdfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:56.587044 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:56.588647 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:56.615040 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a72680-7d3d-44b2-bde1-ec2d7b68fa64 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:56.627144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07811158-6464-40da-b409-b3fc7494cdfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:56.628617 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07811158-6464-40da-b409-b3fc7494cdfb None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:56.647068 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501dab2e-9303-40d9-b3ee-249ec2d06a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:56.647667 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501dab2e-9303-40d9-b3ee-249ec2d06a3a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:56.647952 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501dab2e-9303-40d9-b3ee-249ec2d06a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:56.648225 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501dab2e-9303-40d9-b3ee-249ec2d06a3a None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:56.648968 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501dab2e-9303-40d9-b3ee-249ec2d06a3a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:56.649275 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501dab2e-9303-40d9-b3ee-249ec2d06a3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:56.650667 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501dab2e-9303-40d9-b3ee-249ec2d06a3a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:56.667975 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 225/441] 10.222.0.13 () {56 vars in 1135 bytes} [Sat Oct 22 23:55:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:56.669204 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489c4f4-4414-40fd-889b-907ee56b5411 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:56.669496 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489c4f4-4414-40fd-889b-907ee56b5411 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:56.669768 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489c4f4-4414-40fd-889b-907ee56b5411 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:56.670973 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 226/442] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:55:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:55:56.671667 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489c4f4-4414-40fd-889b-907ee56b5411 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:56.733268 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 217/443] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:55:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 22 23:55:56.734031 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07811158-6464-40da-b409-b3fc7494cdfb None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:56.779653 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489c4f4-4414-40fd-889b-907ee56b5411 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:56.793509 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:56.810323 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:56.816066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861425de-d903-4918-bdde-496ec51f1993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:56.816698 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861425de-d903-4918-bdde-496ec51f1993 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:56.816987 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861425de-d903-4918-bdde-496ec51f1993 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:56.817514 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861425de-d903-4918-bdde-496ec51f1993 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:56.818171 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861425de-d903-4918-bdde-496ec51f1993 None admin] RBAC: Adding query filter params (name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:56.818471 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861425de-d903-4918-bdde-496ec51f1993 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:56.819825 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861425de-d903-4918-bdde-496ec51f1993 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:56.827559 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 227/444] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Oct 22 23:55:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:56.828335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861425de-d903-4918-bdde-496ec51f1993 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:56.837944 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07811158-6464-40da-b409-b3fc7494cdfb None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:56.872225 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef12016-c5e9-45aa-b596-373b536f00bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:56.872875 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef12016-c5e9-45aa-b596-373b536f00bb None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:56.873159 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef12016-c5e9-45aa-b596-373b536f00bb None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:56.873432 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef12016-c5e9-45aa-b596-373b536f00bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:56.884118 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef12016-c5e9-45aa-b596-373b536f00bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:56.885622 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef12016-c5e9-45aa-b596-373b536f00bb None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:56.900618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-6ef12016-c5e9-45aa-b596-373b536f00bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:56.900618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:56.900618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:56.900618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:56.900618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:56.900618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:56.900618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:56.900618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:56.900618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:56.900618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:56.900618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:56.900618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:56.900618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:55:56.900618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:55:56.900618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:55:56.900618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:55:56.901545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:56.901545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:56.901545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:56.901545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:56.901545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:56.901545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:56.901545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:56.901545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:56.901545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:56.901545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:56.901545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:56.901545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:56.901545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:56.901545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:56.901545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:56.901545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:56.901545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:55:56.902416 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:55:56.902416 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:56.902416 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:56.902416 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 218/445] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:56.902912 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d5db2-de0a-4c2e-861b-fa4838e9e2d4 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:56.903093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d5db2-de0a-4c2e-861b-fa4838e9e2d4 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:56.903364 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d5db2-de0a-4c2e-861b-fa4838e9e2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:56.920792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-b30d5db2-de0a-4c2e-861b-fa4838e9e2d4 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:55:56.921975 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b30d5db2-de0a-4c2e-861b-fa4838e9e2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:55:56.928753 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861425de-d903-4918-bdde-496ec51f1993 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:56.962261 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7a233-143c-4688-8016-6297912484e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:56.962878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7a233-143c-4688-8016-6297912484e1 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:56.963160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7a233-143c-4688-8016-6297912484e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:56.963436 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7a233-143c-4688-8016-6297912484e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:56.973924 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a7a233-143c-4688-8016-6297912484e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:56.975346 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a7a233-143c-4688-8016-6297912484e1 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:56.989645 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-c3a7a233-143c-4688-8016-6297912484e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:56.989645 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:56.989645 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:56.989645 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:56.989645 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:56.989645 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:56.989645 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:56.989645 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:56.989645 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:56.989645 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:56.989645 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:56.989645 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:56.989645 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:55:56.989645 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:55:56.989645 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:55:56.989645 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:55:56.990745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:56.990745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:56.990745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:56.990745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:56.990745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:56.990745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:56.990745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:56.990745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:56.990745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:56.990745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:56.990745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:56.990745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:56.990745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:56.990745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:56.990745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:56.990745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:56.990745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:55:56.991816 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:55:56.991816 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:56.991816 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:56.992013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 219/446] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:55:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:55:56.992013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d5db2-de0a-4c2e-861b-fa4838e9e2d4 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:56.992168 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 228/447] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:56.992168 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7a233-143c-4688-8016-6297912484e1 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:57.045562 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:57.054422 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:57.111435 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7a233-143c-4688-8016-6297912484e1 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:57.115362 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d5db2-de0a-4c2e-861b-fa4838e9e2d4 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:57.146334 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86f0f0-1ddb-4a3c-81cc-397e97d6c159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:57.146960 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86f0f0-1ddb-4a3c-81cc-397e97d6c159 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:57.147242 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86f0f0-1ddb-4a3c-81cc-397e97d6c159 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:57.147518 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86f0f0-1ddb-4a3c-81cc-397e97d6c159 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:57.152264 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bf197-c36b-4f98-920f-0ba5f2d092d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:57.153138 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bf197-c36b-4f98-920f-0ba5f2d092d0 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:57.153601 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bf197-c36b-4f98-920f-0ba5f2d092d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:57.154058 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bf197-c36b-4f98-920f-0ba5f2d092d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:57.158898 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e86f0f0-1ddb-4a3c-81cc-397e97d6c159 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:57.160321 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e86f0f0-1ddb-4a3c-81cc-397e97d6c159 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:57.165914 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6bf197-c36b-4f98-920f-0ba5f2d092d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:57.168335 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6bf197-c36b-4f98-920f-0ba5f2d092d0 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:57.174878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-3e86f0f0-1ddb-4a3c-81cc-397e97d6c159 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:57.174878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:57.174878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:57.174878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:57.174878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:57.174878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:57.174878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:57.174878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:57.174878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:57.174878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:57.174878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:57.174878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:57.174878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:55:57.174878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:55:57.174878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:55:57.174878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:55:57.175791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:57.175791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:57.175791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:57.175791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:57.175791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:57.175791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:57.175791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:57.175791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:57.175791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:57.175791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:57.175791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:57.175791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:57.175791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:57.175791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:57.175791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:57.175791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:57.175791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:55:57.176661 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:55:57.176661 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:57.176661 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:57.176661 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 229/448] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:57.177613 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86f0f0-1ddb-4a3c-81cc-397e97d6c159 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:57.187030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-af6bf197-c36b-4f98-920f-0ba5f2d092d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:57.187030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:57.187030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:57.187030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:57.187030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:57.187030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:57.187030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:57.187030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:57.187030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:57.187030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:57.187030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:57.187030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:57.187030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:55:57.187030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:55:57.187030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:55:57.187030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:55:57.187943 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:57.187943 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:57.187943 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:57.187943 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:57.187943 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:57.187943 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:57.187943 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:57.187943 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:57.187943 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:57.187943 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:57.187943 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:57.187943 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:57.187943 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:57.187943 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:57.187943 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:57.187943 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:57.187943 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:55:57.189042 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:55:57.189042 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:57.189042 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:57.192897 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 220/449] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:57.193892 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bf197-c36b-4f98-920f-0ba5f2d092d0 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:57.279650 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:57.284215 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86f0f0-1ddb-4a3c-81cc-397e97d6c159 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:57.293857 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:57.300818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bf197-c36b-4f98-920f-0ba5f2d092d0 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:57.319599 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9df52-5b0c-4cca-8b48-bc0560aca208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:57.320210 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9df52-5b0c-4cca-8b48-bc0560aca208 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:57.320508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9df52-5b0c-4cca-8b48-bc0560aca208 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:57.320806 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da9df52-5b0c-4cca-8b48-bc0560aca208 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:57.321446 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9df52-5b0c-4cca-8b48-bc0560aca208 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:57.321748 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9df52-5b0c-4cca-8b48-bc0560aca208 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:57.323114 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da9df52-5b0c-4cca-8b48-bc0560aca208 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:57.335142 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 230/450] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:57.335263 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f68f1a-ae45-42db-9b7a-a05ade02d694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:57.335775 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9df52-5b0c-4cca-8b48-bc0560aca208 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:57.335908 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f68f1a-ae45-42db-9b7a-a05ade02d694 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:57.336134 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f68f1a-ae45-42db-9b7a-a05ade02d694 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:57.336416 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f68f1a-ae45-42db-9b7a-a05ade02d694 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:57.337092 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f68f1a-ae45-42db-9b7a-a05ade02d694 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:57.337398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f68f1a-ae45-42db-9b7a-a05ade02d694 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:57.338766 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f68f1a-ae45-42db-9b7a-a05ade02d694 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:57.350842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 221/451] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:57.350842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f68f1a-ae45-42db-9b7a-a05ade02d694 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:57.436556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da9df52-5b0c-4cca-8b48-bc0560aca208 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:57.448132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f68f1a-ae45-42db-9b7a-a05ade02d694 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:57.472457 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40dfb3a-52f0-45a8-ba91-845222a401b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:57.473084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40dfb3a-52f0-45a8-ba91-845222a401b7 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:57.473370 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40dfb3a-52f0-45a8-ba91-845222a401b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:57.473679 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40dfb3a-52f0-45a8-ba91-845222a401b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:57.474485 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40dfb3a-52f0-45a8-ba91-845222a401b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:57.475118 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40dfb3a-52f0-45a8-ba91-845222a401b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:57.476513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40dfb3a-52f0-45a8-ba91-845222a401b7 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:57.481987 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9643e4-719c-4292-9a2f-43e1dbbd9ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:57.482579 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9643e4-719c-4292-9a2f-43e1dbbd9ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:57.482862 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9643e4-719c-4292-9a2f-43e1dbbd9ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:57.483133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9643e4-719c-4292-9a2f-43e1dbbd9ec2 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:57.483754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9643e4-719c-4292-9a2f-43e1dbbd9ec2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:57.484075 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9643e4-719c-4292-9a2f-43e1dbbd9ec2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:57.485748 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9643e4-719c-4292-9a2f-43e1dbbd9ec2 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:57.487772 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 231/452] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:57.488690 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40dfb3a-52f0-45a8-ba91-845222a401b7 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:57.497635 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 222/453] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:57.498711 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9643e4-719c-4292-9a2f-43e1dbbd9ec2 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:57.512426 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:57.518350 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:57.593528 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40dfb3a-52f0-45a8-ba91-845222a401b7 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:57.598762 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9643e4-719c-4292-9a2f-43e1dbbd9ec2 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:57.626174 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef0ebe-a347-4fea-9924-6a12a7636062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:57.626929 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef0ebe-a347-4fea-9924-6a12a7636062 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:57.627213 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef0ebe-a347-4fea-9924-6a12a7636062 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:57.627482 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef0ebe-a347-4fea-9924-6a12a7636062 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:57.630954 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43af09-27fb-41bc-8411-331b7691149f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:57.631576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43af09-27fb-41bc-8411-331b7691149f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:57.631889 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43af09-27fb-41bc-8411-331b7691149f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:57.632168 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b43af09-27fb-41bc-8411-331b7691149f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:57.637567 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef0ebe-a347-4fea-9924-6a12a7636062 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:55:57.639286 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43af09-27fb-41bc-8411-331b7691149f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:57.640751 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b43af09-27fb-41bc-8411-331b7691149f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:57.643451 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-d7ef0ebe-a347-4fea-9924-6a12a7636062 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:55:57.643451 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:57.643451 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:57.643451 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:57.643451 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:57.643451 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:57.643451 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:57.643451 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:57.643451 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:57.643451 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:57.643451 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:57.643451 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:57.643451 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:55:57.643451 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:55:57.643451 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:55:57.643451 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:55:57.644354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:55:57.644354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:55:57.644354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:55:57.644354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:55:57.644354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:57.644354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:57.644354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:57.644354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:57.644354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:57.644354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:57.644354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:57.644354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:57.644354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:57.644354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:57.644354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:57.644354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:57.646380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:57.646380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:57.646380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:57.646380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:57.646380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:55:57.646380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:55:57.646380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:55:57.646380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:55:57.646380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:55:57.646380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:55:57.646380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:55:57.646380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:57.646380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 232/454] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Oct 22 23:55:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:57.646380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef0ebe-a347-4fea-9924-6a12a7636062 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:57.653277 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-8b43af09-27fb-41bc-8411-331b7691149f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 22 23:55:57.653277 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:57.653277 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:57.653277 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:57.653277 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:57.653277 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:57.653277 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:57.653277 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:57.653277 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:57.653277 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:57.653277 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:57.653277 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:57.653277 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:55:57.653277 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:55:57.653277 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:55:57.653277 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:55:57.654183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:57.654183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:57.654183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:55:57.654183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:55:57.654183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:55:57.654183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:55:57.654183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:57.654183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:57.654183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:57.654183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:57.654183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:57.654183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:57.654183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:57.654183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:57.654183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:57.654183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:57.654183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:57.654183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:57.655070 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:57.655070 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:57.655070 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:55:57.655070 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:55:57.655070 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:55:57.655070 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:55:57.655070 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:55:57.655070 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:55:57.655070 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 22 23:55:57.655070 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:57.655070 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 223/455] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:57.655628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43af09-27fb-41bc-8411-331b7691149f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:57.726026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:57.731535 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:57.748865 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef0ebe-a347-4fea-9924-6a12a7636062 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:57.756513 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b43af09-27fb-41bc-8411-331b7691149f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:57.781791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32556358-dbea-47f9-bb0c-395f18b2009e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:57.782492 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32556358-dbea-47f9-bb0c-395f18b2009e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:57.782773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32556358-dbea-47f9-bb0c-395f18b2009e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:57.783054 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32556358-dbea-47f9-bb0c-395f18b2009e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:57.789596 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32556358-dbea-47f9-bb0c-395f18b2009e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:57.790253 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70ab11-3ed9-47a0-b282-ff6139e1a3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:57.790873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70ab11-3ed9-47a0-b282-ff6139e1a3c6 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:57.791041 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32556358-dbea-47f9-bb0c-395f18b2009e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:57.791157 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70ab11-3ed9-47a0-b282-ff6139e1a3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:57.791434 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70ab11-3ed9-47a0-b282-ff6139e1a3c6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:57.799160 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af70ab11-3ed9-47a0-b282-ff6139e1a3c6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:57.800600 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af70ab11-3ed9-47a0-b282-ff6139e1a3c6 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:57.802773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-32556358-dbea-47f9-bb0c-395f18b2009e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:55:57.802773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:57.802773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:57.802773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:57.802773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:57.802773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:57.802773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:57.802773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:57.802773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:57.802773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:57.802773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:57.802773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:57.802773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:55:57.802773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:55:57.802773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:55:57.803740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:55:57.803740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:57.803740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:57.803740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:57.803740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:57.803740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:57.803740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:57.803740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:57.803740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:57.803740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:57.803740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:57.803740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:57.803740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:57.803740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:57.803740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:57.803740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:57.803740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:57.804779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:55:57.804779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:55:57.804779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:55:57.804779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:55:57.804779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:55:57.804779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:55:57.804779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:55:57.804779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:57.804779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 233/456] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:55:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:57.804779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32556358-dbea-47f9-bb0c-395f18b2009e None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:57.814650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-af70ab11-3ed9-47a0-b282-ff6139e1a3c6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 22 23:55:57.814650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:57.814650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:57.814650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:57.814650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:57.814650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:57.814650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:57.814650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:57.814650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:57.814650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:57.814650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:57.814650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:57.814650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:55:57.814650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:55:57.814650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:55:57.814650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:55:57.815580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:57.815580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:57.815580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:55:57.815580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:55:57.815580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:55:57.815580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:55:57.815580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:57.815580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:57.815580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:57.815580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:57.815580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:57.815580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:57.815580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:57.815580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:57.815580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:57.815580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:57.815580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:57.815580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:57.816594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:57.816594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:57.816594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:55:57.816594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:55:57.816594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:55:57.816594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:55:57.816594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:55:57.816594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:55:57.816594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 22 23:55:57.816594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:57.816594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 224/457] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:55:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:57.817422 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70ab11-3ed9-47a0-b282-ff6139e1a3c6 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:57.908318 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32556358-dbea-47f9-bb0c-395f18b2009e None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:57.914136 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70ab11-3ed9-47a0-b282-ff6139e1a3c6 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:57.941424 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12001095-cbef-41c2-8299-47b8d14ce311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:57.942473 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12001095-cbef-41c2-8299-47b8d14ce311 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:57.942768 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12001095-cbef-41c2-8299-47b8d14ce311 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:57.943043 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12001095-cbef-41c2-8299-47b8d14ce311 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:57.947294 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731675d3-96d1-4b0e-9420-1eba2bec7e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:57.947893 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731675d3-96d1-4b0e-9420-1eba2bec7e4d None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:57.948180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731675d3-96d1-4b0e-9420-1eba2bec7e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:57.948461 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731675d3-96d1-4b0e-9420-1eba2bec7e4d None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:57.949155 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731675d3-96d1-4b0e-9420-1eba2bec7e4d None admin] RBAC: Adding query filter params (name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:57.949463 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731675d3-96d1-4b0e-9420-1eba2bec7e4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:57.949552 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12001095-cbef-41c2-8299-47b8d14ce311 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:57.950831 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731675d3-96d1-4b0e-9420-1eba2bec7e4d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:57.951024 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12001095-cbef-41c2-8299-47b8d14ce311 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:57.952699 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:57.953290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:57.964813 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 225/458] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Oct 22 23:55:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:57.964813 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731675d3-96d1-4b0e-9420-1eba2bec7e4d None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:57.969508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-12001095-cbef-41c2-8299-47b8d14ce311 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:55:57.969508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:57.969508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:57.969508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:57.969508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:57.969508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:57.969508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:57.969508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:57.969508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:57.969508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:57.969508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:57.969508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:57.969508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:55:57.969508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:55:57.969508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:55:57.970915 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:55:57.970915 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:57.970915 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:57.970915 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:57.970915 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:57.970915 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:57.970915 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:57.970915 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:57.970915 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:57.970915 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:57.970915 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:57.970915 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:57.970915 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:57.970915 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:57.970915 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:57.970915 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:57.970915 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:57.971979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:55:57.971979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:55:57.971979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:55:57.971979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:55:57.971979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:55:57.971979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:55:57.971979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:55:57.971979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:57.971979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 234/459] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:55:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:57.971979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12001095-cbef-41c2-8299-47b8d14ce311 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:58.063229 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731675d3-96d1-4b0e-9420-1eba2bec7e4d None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:58.067770 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12001095-cbef-41c2-8299-47b8d14ce311 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:58.096400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0cc05f-375b-44c4-8936-10b8eb96c542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:58.097041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0cc05f-375b-44c4-8936-10b8eb96c542 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:58.097328 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0cc05f-375b-44c4-8936-10b8eb96c542 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:58.097666 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0cc05f-375b-44c4-8936-10b8eb96c542 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:58.098328 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0cc05f-375b-44c4-8936-10b8eb96c542 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:58.098642 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0cc05f-375b-44c4-8936-10b8eb96c542 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:58.100059 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0cc05f-375b-44c4-8936-10b8eb96c542 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:58.100460 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f014288-f549-4de6-a7ab-13711fdf1668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:58.103694 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f014288-f549-4de6-a7ab-13711fdf1668 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:58.103694 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f014288-f549-4de6-a7ab-13711fdf1668 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:58.103694 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f014288-f549-4de6-a7ab-13711fdf1668 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:58.103694 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f014288-f549-4de6-a7ab-13711fdf1668 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:58.103694 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f014288-f549-4de6-a7ab-13711fdf1668 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:58.104101 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f014288-f549-4de6-a7ab-13711fdf1668 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:58.119259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 235/460] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Oct 22 23:55:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:58.120170 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f014288-f549-4de6-a7ab-13711fdf1668 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:58.125096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 226/461] 10.222.0.13 () {56 vars in 1137 bytes} [Sat Oct 22 23:55:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:58.125096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0cc05f-375b-44c4-8936-10b8eb96c542 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:58.178344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:58.181707 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:58.222880 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0cc05f-375b-44c4-8936-10b8eb96c542 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:58.223046 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f014288-f549-4de6-a7ab-13711fdf1668 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:58.256112 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeab832-5fa0-4c27-af54-b874f54f5036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:58.256525 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23eb68-c657-42f2-9911-02ceae4753ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:58.256897 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeab832-5fa0-4c27-af54-b874f54f5036 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:58.257155 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23eb68-c657-42f2-9911-02ceae4753ba None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:58.257337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeab832-5fa0-4c27-af54-b874f54f5036 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:58.257433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23eb68-c657-42f2-9911-02ceae4753ba None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:58.257707 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23eb68-c657-42f2-9911-02ceae4753ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:58.257793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeab832-5fa0-4c27-af54-b874f54f5036 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:58.258531 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeab832-5fa0-4c27-af54-b874f54f5036 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:58.258891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeab832-5fa0-4c27-af54-b874f54f5036 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:58.260360 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeab832-5fa0-4c27-af54-b874f54f5036 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:58.267872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23eb68-c657-42f2-9911-02ceae4753ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:58.269322 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23eb68-c657-42f2-9911-02ceae4753ba None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:58.277551 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 236/462] 10.222.0.13 () {56 vars in 1137 bytes} [Sat Oct 22 23:55:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:58.278530 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeab832-5fa0-4c27-af54-b874f54f5036 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:58.284792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-9b23eb68-c657-42f2-9911-02ceae4753ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:58.284792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:58.284792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:58.284792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:58.284792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:58.284792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:58.284792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:58.284792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:58.284792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:58.284792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:58.284792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:58.284792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:58.284792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:55:58.284792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:55:58.284792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:55:58.284792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:55:58.285853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:58.285853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:58.285853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:58.285853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:58.285853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:58.285853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:58.285853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:58.285853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:58.285853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:58.285853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:58.285853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:58.285853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:58.285853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:58.285853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:58.285853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:58.285853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:58.285853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:55:58.286711 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:55:58.286711 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:58.286711 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:58.286711 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 227/463] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:58.286711 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23eb68-c657-42f2-9911-02ceae4753ba None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:58.378468 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeab832-5fa0-4c27-af54-b874f54f5036 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:58.383619 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23eb68-c657-42f2-9911-02ceae4753ba None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:58.394100 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:58.394963 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:58.412709 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425887ac-4428-4d2d-98eb-603ea31dad17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:58.413314 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425887ac-4428-4d2d-98eb-603ea31dad17 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:58.413600 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425887ac-4428-4d2d-98eb-603ea31dad17 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:58.413878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425887ac-4428-4d2d-98eb-603ea31dad17 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:58.414725 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425887ac-4428-4d2d-98eb-603ea31dad17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:58.415352 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425887ac-4428-4d2d-98eb-603ea31dad17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:58.416804 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425887ac-4428-4d2d-98eb-603ea31dad17 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:58.419436 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1486593-10ba-42c2-b9cd-e4fdd3a5ae20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:58.420076 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1486593-10ba-42c2-b9cd-e4fdd3a5ae20 None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:58.420355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1486593-10ba-42c2-b9cd-e4fdd3a5ae20 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:58.420665 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1486593-10ba-42c2-b9cd-e4fdd3a5ae20 None admin] PATH_INFO: `/v3/projects/5c9246c059754504afee4711d113a3fe/users/bfd8c7516b7c435d88298672b2596d6d/roles/73721a054c3842bba868ab52c1a3f8e0` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:58.429408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 237/464] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Oct 22 23:55:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:58.430214 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425887ac-4428-4d2d-98eb-603ea31dad17 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:58.440926 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1486593-10ba-42c2-b9cd-e4fdd3a5ae20 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c9246c059754504afee4711d113a3fe, user_id=bfd8c7516b7c435d88298672b2596d6d, role_id=73721a054c3842bba868ab52c1a3f8e0)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:58.442649 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1486593-10ba-42c2-b9cd-e4fdd3a5ae20 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:58.455088 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 228/465] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:55:58 2022] PUT /identity/v3/projects/5c9246c059754504afee4711d113a3fe/users/bfd8c7516b7c435d88298672b2596d6d/roles/73721a054c3842bba868ab52c1a3f8e0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:55:58.455888 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1486593-10ba-42c2-b9cd-e4fdd3a5ae20 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:58.542272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425887ac-4428-4d2d-98eb-603ea31dad17 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:58.579455 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947bef6a-04f8-4d4d-a2de-e917e1599818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:58.580042 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bef6a-04f8-4d4d-a2de-e917e1599818 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:58.580328 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bef6a-04f8-4d4d-a2de-e917e1599818 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:58.580651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bef6a-04f8-4d4d-a2de-e917e1599818 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:58.585597 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1486593-10ba-42c2-b9cd-e4fdd3a5ae20 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:58.621777 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb47269-221f-4b68-8d97-b245c6036847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:58.622446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb47269-221f-4b68-8d97-b245c6036847 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:58.622656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb47269-221f-4b68-8d97-b245c6036847 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:58.622942 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb47269-221f-4b68-8d97-b245c6036847 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:58.631489 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:58.653893 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:58.666938 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947bef6a-04f8-4d4d-a2de-e917e1599818 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:58.668399 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947bef6a-04f8-4d4d-a2de-e917e1599818 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:58.698545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb47269-221f-4b68-8d97-b245c6036847 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:58.699989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb47269-221f-4b68-8d97-b245c6036847 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:58.775806 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 238/466] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:55:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 22 23:55:58.777430 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947bef6a-04f8-4d4d-a2de-e917e1599818 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:58.809007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 229/467] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:55:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 22 23:55:58.809936 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb47269-221f-4b68-8d97-b245c6036847 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:58.866008 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:58.875580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:58.894091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947bef6a-04f8-4d4d-a2de-e917e1599818 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:58.925766 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb47269-221f-4b68-8d97-b245c6036847 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:58.933469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f25521-183a-4577-a952-5e0d2d609d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:58.934098 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f25521-183a-4577-a952-5e0d2d609d9c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:58.934382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f25521-183a-4577-a952-5e0d2d609d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:58.934691 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f25521-183a-4577-a952-5e0d2d609d9c None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:58.935337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f25521-183a-4577-a952-5e0d2d609d9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:58.935644 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f25521-183a-4577-a952-5e0d2d609d9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:58.937093 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f25521-183a-4577-a952-5e0d2d609d9c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:58.951215 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 239/468] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:58.952973 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f25521-183a-4577-a952-5e0d2d609d9c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:58.964310 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c9e26-8abe-4e44-896c-4835adbe2f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:58.965143 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c9e26-8abe-4e44-896c-4835adbe2f88 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:58.966041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c9e26-8abe-4e44-896c-4835adbe2f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:58.966419 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c9e26-8abe-4e44-896c-4835adbe2f88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:58.967244 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c9e26-8abe-4e44-896c-4835adbe2f88 None admin] RBAC: Adding query filter params (role.id=73721a054c3842bba868ab52c1a3f8e0, scope.project.id=5c9246c059754504afee4711d113a3fe, user.id=c4a3579d56294b9d8ad36d4d174c8526) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:58.968055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c9e26-8abe-4e44-896c-4835adbe2f88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:58.970572 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c9e26-8abe-4e44-896c-4835adbe2f88 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:58.979158 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 230/469] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:55:58 2022] GET /identity/v3/role_assignments?user.id=c4a3579d56294b9d8ad36d4d174c8526&scope.project.id=5c9246c059754504afee4711d113a3fe&role.id=73721a054c3842bba868ab52c1a3f8e0 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:58.980166 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c9e26-8abe-4e44-896c-4835adbe2f88 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:59.091533 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f25521-183a-4577-a952-5e0d2d609d9c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:59.093994 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c9e26-8abe-4e44-896c-4835adbe2f88 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:59.136143 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:59.140418 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105343bd-dc17-40b3-9537-2cc7e4713f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:59.141174 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105343bd-dc17-40b3-9537-2cc7e4713f6e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:59.141464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105343bd-dc17-40b3-9537-2cc7e4713f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:59.142301 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105343bd-dc17-40b3-9537-2cc7e4713f6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:59.152966 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:59.153148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957be344-5703-464b-b26c-147b79ea6747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:59.153712 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957be344-5703-464b-b26c-147b79ea6747 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:59.154047 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957be344-5703-464b-b26c-147b79ea6747 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:59.154342 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957be344-5703-464b-b26c-147b79ea6747 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:59.158497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105343bd-dc17-40b3-9537-2cc7e4713f6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:59.159986 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105343bd-dc17-40b3-9537-2cc7e4713f6e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:59.171716 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957be344-5703-464b-b26c-147b79ea6747 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:59.173882 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957be344-5703-464b-b26c-147b79ea6747 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:59.181631 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-105343bd-dc17-40b3-9537-2cc7e4713f6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:59.181631 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:59.181631 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:59.181631 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:59.181631 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:59.181631 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:59.181631 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:59.181631 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:59.181631 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:59.181631 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:59.181631 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:59.181631 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:59.181631 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:55:59.181631 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:55:59.181631 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:55:59.181631 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:55:59.182573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:59.182573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:59.182573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:59.182573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:59.182573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:59.182573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:59.182573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:59.182573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:59.182573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:59.182573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:59.182573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:59.182573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:59.182573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:59.182573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:59.182573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:59.182573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:59.182573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:55:59.183429 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:55:59.183429 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:59.183429 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:59.183429 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 240/470] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:59.184077 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105343bd-dc17-40b3-9537-2cc7e4713f6e None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:59.198854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-957be344-5703-464b-b26c-147b79ea6747 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:59.198854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:59.198854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:59.198854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:59.198854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:59.198854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:59.198854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:59.198854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:59.198854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:59.198854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:59.198854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:59.198854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:59.198854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:55:59.198854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:55:59.198854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:55:59.198854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:55:59.199823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:59.199823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:59.199823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:59.199823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:59.199823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:59.199823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:59.199823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:59.199823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:59.199823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:59.199823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:59.199823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:59.199823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:59.199823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:59.199823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:59.199823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:59.199823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:59.199823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:55:59.201239 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:55:59.201239 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:55:59.201239 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:55:59.201239 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 231/471] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:55:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:59.201239 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957be344-5703-464b-b26c-147b79ea6747 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:59.317908 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105343bd-dc17-40b3-9537-2cc7e4713f6e None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:59.319852 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957be344-5703-464b-b26c-147b79ea6747 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:59.362469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f08012-62df-49f8-adb6-4bd4ed155c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:59.363431 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f08012-62df-49f8-adb6-4bd4ed155c1e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:59.363921 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f08012-62df-49f8-adb6-4bd4ed155c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:59.364318 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7f14b-3df1-4e7c-91e9-b79f0c491039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:59.364672 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f08012-62df-49f8-adb6-4bd4ed155c1e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:59.364966 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7f14b-3df1-4e7c-91e9-b79f0c491039 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:59.365248 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7f14b-3df1-4e7c-91e9-b79f0c491039 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:59.365524 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7f14b-3df1-4e7c-91e9-b79f0c491039 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:59.366171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7f14b-3df1-4e7c-91e9-b79f0c491039 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:59.366474 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7f14b-3df1-4e7c-91e9-b79f0c491039 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:59.367849 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7f14b-3df1-4e7c-91e9-b79f0c491039 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:59.374206 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f08012-62df-49f8-adb6-4bd4ed155c1e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:59.376460 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f08012-62df-49f8-adb6-4bd4ed155c1e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:59.383195 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 232/472] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:59.384338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7f14b-3df1-4e7c-91e9-b79f0c491039 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:59.390802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-d7f08012-62df-49f8-adb6-4bd4ed155c1e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 22 23:55:59.390802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:59.390802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:59.390802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:59.390802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:59.390802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:59.390802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:59.390802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:59.390802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:59.390802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:59.390802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:59.390802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:59.390802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:55:59.390802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:55:59.390802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:55:59.390802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:55:59.391976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:59.391976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:59.391976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:55:59.391976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:55:59.391976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:55:59.391976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:55:59.391976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:59.391976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:59.391976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:59.391976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:59.391976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:59.391976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:59.391976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:59.391976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:59.391976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:59.391976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:59.391976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:59.391976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:59.392940 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:59.392940 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:59.392940 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:55:59.392940 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:55:59.392940 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:55:59.392940 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:55:59.392940 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:55:59.392940 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:55:59.392940 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 22 23:55:59.392940 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:59.392940 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 241/473] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Oct 22 23:55:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:59.392940 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f08012-62df-49f8-adb6-4bd4ed155c1e None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:59.429820 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:59.461537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:59.518969 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7f14b-3df1-4e7c-91e9-b79f0c491039 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:59.527175 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f08012-62df-49f8-adb6-4bd4ed155c1e None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:59.557224 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561c047-a3c8-4fe3-a119-b23c40fff433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:59.557666 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561c047-a3c8-4fe3-a119-b23c40fff433 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:59.557952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561c047-a3c8-4fe3-a119-b23c40fff433 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:59.558238 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561c047-a3c8-4fe3-a119-b23c40fff433 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:59.558896 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561c047-a3c8-4fe3-a119-b23c40fff433 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:59.559188 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561c047-a3c8-4fe3-a119-b23c40fff433 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:59.560656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561c047-a3c8-4fe3-a119-b23c40fff433 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:59.570676 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd88c9-d07f-408e-b143-03715f4b1f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:59.571902 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd88c9-d07f-408e-b143-03715f4b1f8e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:59.572382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd88c9-d07f-408e-b143-03715f4b1f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:59.572885 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd88c9-d07f-408e-b143-03715f4b1f8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:59.577836 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 233/474] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:59.577836 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561c047-a3c8-4fe3-a119-b23c40fff433 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:59.587831 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd88c9-d07f-408e-b143-03715f4b1f8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:59.590281 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd88c9-d07f-408e-b143-03715f4b1f8e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:59.604648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-fadd88c9-d07f-408e-b143-03715f4b1f8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:55:59.604648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:59.604648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:59.604648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:59.604648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:59.604648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:59.604648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:59.604648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:59.604648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:59.604648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:59.604648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:59.604648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:59.604648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:55:59.604648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:55:59.604648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:55:59.605513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:55:59.605513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:59.605513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:59.605513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:59.605513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:59.605513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:59.605513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:59.605513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:59.605513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:59.605513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:59.605513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:59.605513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:59.605513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:59.605513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:59.605513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:59.605513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:59.605513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:59.606351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:55:59.606351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:55:59.606351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:55:59.606351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:55:59.606351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:55:59.606351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:55:59.606351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:55:59.606351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:59.609397 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 242/475] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:55:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:59.610824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd88c9-d07f-408e-b143-03715f4b1f8e None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:59.677005 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:59.685519 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561c047-a3c8-4fe3-a119-b23c40fff433 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:59.722580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459438d6-53c1-4745-b8b6-7297f9c5c889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:59.723201 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459438d6-53c1-4745-b8b6-7297f9c5c889 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:59.723491 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459438d6-53c1-4745-b8b6-7297f9c5c889 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:59.723778 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459438d6-53c1-4745-b8b6-7297f9c5c889 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:59.724543 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459438d6-53c1-4745-b8b6-7297f9c5c889 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:59.724863 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459438d6-53c1-4745-b8b6-7297f9c5c889 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:59.726304 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459438d6-53c1-4745-b8b6-7297f9c5c889 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:59.733742 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd88c9-d07f-408e-b143-03715f4b1f8e None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:59.749557 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:55:59.751098 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 234/476] 10.222.0.13 () {56 vars in 1139 bytes} [Sat Oct 22 23:55:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:59.752581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459438d6-53c1-4745-b8b6-7297f9c5c889 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:59.781263 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cebe9a-fdac-4df7-aa57-05977332fe09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:59.782404 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cebe9a-fdac-4df7-aa57-05977332fe09 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:59.782914 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cebe9a-fdac-4df7-aa57-05977332fe09 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:59.783420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cebe9a-fdac-4df7-aa57-05977332fe09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:59.792194 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cebe9a-fdac-4df7-aa57-05977332fe09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:59.794022 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cebe9a-fdac-4df7-aa57-05977332fe09 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:59.808537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-54cebe9a-fdac-4df7-aa57-05977332fe09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:55:59.808537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:55:59.808537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:55:59.808537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:55:59.808537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:55:59.808537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:55:59.808537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:55:59.808537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:55:59.808537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:55:59.808537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:55:59.808537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:55:59.808537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:55:59.808537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:55:59.808537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:55:59.808537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:55:59.809447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:55:59.809447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:55:59.809447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:55:59.809447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:55:59.809447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:55:59.809447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:55:59.809447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:55:59.809447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:55:59.809447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:55:59.809447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:55:59.809447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:55:59.809447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:55:59.809447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:55:59.809447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:55:59.809447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:55:59.809447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:55:59.809447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:55:59.810304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:55:59.810304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:55:59.810304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:55:59.810304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:55:59.810304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:55:59.810304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:55:59.810304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:55:59.810304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:55:59.816802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 243/477] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:55:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:55:59.818447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cebe9a-fdac-4df7-aa57-05977332fe09 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:59.872117 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459438d6-53c1-4745-b8b6-7297f9c5c889 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:59.914760 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c6e01-94d0-48fa-a0ea-dff8eb0008ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:55:59.915368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c6e01-94d0-48fa-a0ea-dff8eb0008ac None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:55:59.915793 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c6e01-94d0-48fa-a0ea-dff8eb0008ac None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:55:59.916105 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c6e01-94d0-48fa-a0ea-dff8eb0008ac None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:55:59.917577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c6e01-94d0-48fa-a0ea-dff8eb0008ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:55:59.917898 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c6e01-94d0-48fa-a0ea-dff8eb0008ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:55:59.919344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c6e01-94d0-48fa-a0ea-dff8eb0008ac None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:55:59.934362 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 235/478] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Oct 22 23:55:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:55:59.935209 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c6e01-94d0-48fa-a0ea-dff8eb0008ac None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:55:59.953412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cebe9a-fdac-4df7-aa57-05977332fe09 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:55:59.955537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:00.002980 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dcc9aa-d1da-4fd6-bbdb-fe4547093651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:00.003595 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcc9aa-d1da-4fd6-bbdb-fe4547093651 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:00.003896 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcc9aa-d1da-4fd6-bbdb-fe4547093651 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:00.004450 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcc9aa-d1da-4fd6-bbdb-fe4547093651 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:00.042433 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:00.064855 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c6e01-94d0-48fa-a0ea-dff8eb0008ac None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:00.080933 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dcc9aa-d1da-4fd6-bbdb-fe4547093651 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:00.082735 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dcc9aa-d1da-4fd6-bbdb-fe4547093651 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:00.108591 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9c635-7d36-42f8-9787-eadb5c3b5416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:00.109767 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9c635-7d36-42f8-9787-eadb5c3b5416 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:00.110052 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9c635-7d36-42f8-9787-eadb5c3b5416 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:00.111479 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9c635-7d36-42f8-9787-eadb5c3b5416 None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:00.112197 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9c635-7d36-42f8-9787-eadb5c3b5416 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:00.112530 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9c635-7d36-42f8-9787-eadb5c3b5416 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:00.114126 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9c635-7d36-42f8-9787-eadb5c3b5416 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:00.132953 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 236/479] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Oct 22 23:55:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:00.133899 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9c635-7d36-42f8-9787-eadb5c3b5416 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:00.208618 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 244/480] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:55:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 22 23:56:00.210005 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dcc9aa-d1da-4fd6-bbdb-fe4547093651 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:00.211316 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:00.248578 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9c635-7d36-42f8-9787-eadb5c3b5416 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:00.284400 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:00.287723 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66d364-3473-49b1-8bf7-49dec5919c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:00.288726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66d364-3473-49b1-8bf7-49dec5919c26 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:00.290970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66d364-3473-49b1-8bf7-49dec5919c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:00.291407 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66d364-3473-49b1-8bf7-49dec5919c26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:00.292285 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66d364-3473-49b1-8bf7-49dec5919c26 None admin] RBAC: Adding query filter params (role.id=73721a054c3842bba868ab52c1a3f8e0, scope.project.id=5c9246c059754504afee4711d113a3fe, user.id=e4a69aeb3fba49b6881f1f15c1758494) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:00.292794 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66d364-3473-49b1-8bf7-49dec5919c26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:00.294395 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf66d364-3473-49b1-8bf7-49dec5919c26 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:00.302708 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 237/481] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:56:00 2022] GET /identity/v3/role_assignments?user.id=e4a69aeb3fba49b6881f1f15c1758494&scope.project.id=5c9246c059754504afee4711d113a3fe&role.id=73721a054c3842bba868ab52c1a3f8e0 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:00.303850 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66d364-3473-49b1-8bf7-49dec5919c26 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:00.327207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dcc9aa-d1da-4fd6-bbdb-fe4547093651 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:00.370993 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905befb-c630-4242-8868-f4717450e2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:00.371764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905befb-c630-4242-8868-f4717450e2c3 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:00.372257 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905befb-c630-4242-8868-f4717450e2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:00.372561 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905befb-c630-4242-8868-f4717450e2c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:00.373481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905befb-c630-4242-8868-f4717450e2c3 None admin] RBAC: Adding query filter params (role.id=73721a054c3842bba868ab52c1a3f8e0, scope.project.id=5c9246c059754504afee4711d113a3fe, user.id=bd170de478a344fc98951697c7a6298d) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:00.373829 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905befb-c630-4242-8868-f4717450e2c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:00.375548 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905befb-c630-4242-8868-f4717450e2c3 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:00.384532 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 245/482] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:56:00 2022] GET /identity/v3/role_assignments?user.id=bd170de478a344fc98951697c7a6298d&scope.project.id=5c9246c059754504afee4711d113a3fe&role.id=73721a054c3842bba868ab52c1a3f8e0 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:00.464043 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66d364-3473-49b1-8bf7-49dec5919c26 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:00.514922 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c754c56-9929-4955-b3c5-103ef54bee6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:00.515572 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c754c56-9929-4955-b3c5-103ef54bee6e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:00.515857 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c754c56-9929-4955-b3c5-103ef54bee6e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:00.516133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c754c56-9929-4955-b3c5-103ef54bee6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:00.523470 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:00.537171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c754c56-9929-4955-b3c5-103ef54bee6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:00.538661 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c754c56-9929-4955-b3c5-103ef54bee6e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:00.557391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-0c754c56-9929-4955-b3c5-103ef54bee6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:00.557391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:00.557391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:00.557391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:00.557391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:00.557391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:00.557391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:00.557391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:00.557391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:00.557391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:00.557391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:00.557391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:00.557391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:00.557391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:00.557391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:00.557391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:00.558378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:00.558378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:00.558378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:00.558378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:00.558378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:00.558378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:00.558378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:00.558378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:00.558378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:00.558378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:00.558378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:00.558378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:00.558378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:00.558378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:00.558378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:00.558378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:00.558378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:00.559240 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:00.559240 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:00.559240 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:00.559240 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 238/483] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:00.564608 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905befb-c630-4242-8868-f4717450e2c3 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:00.699120 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905befb-c630-4242-8868-f4717450e2c3 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:00.730788 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:00.736844 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3fdb5d-142b-43ae-b45d-217ae201369e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:00.737627 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3fdb5d-142b-43ae-b45d-217ae201369e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:00.737979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3fdb5d-142b-43ae-b45d-217ae201369e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:00.738417 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3fdb5d-142b-43ae-b45d-217ae201369e None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:00.739311 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3fdb5d-142b-43ae-b45d-217ae201369e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:00.739620 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3fdb5d-142b-43ae-b45d-217ae201369e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:00.741032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3fdb5d-142b-43ae-b45d-217ae201369e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:00.754721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 246/484] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:00.759996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c754c56-9929-4955-b3c5-103ef54bee6e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:00.774546 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffb455-2022-463e-b05a-e2d48c7a6016 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:00.774844 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffb455-2022-463e-b05a-e2d48c7a6016 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:00.775107 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffb455-2022-463e-b05a-e2d48c7a6016 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:00.776446 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 247/485] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:00.796780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d1f50-8467-4df4-88c7-4e094eb01b98 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:00.797099 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d1f50-8467-4df4-88c7-4e094eb01b98 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:00.797348 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d1f50-8467-4df4-88c7-4e094eb01b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:00.817515 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-2f0d1f50-8467-4df4-88c7-4e094eb01b98 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:00.818730 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-2f0d1f50-8467-4df4-88c7-4e094eb01b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:00.894901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 248/486] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 22 23:56:00.897914 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c754c56-9929-4955-b3c5-103ef54bee6e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:00.933625 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d7cbb-b6ed-4290-b654-cc6bca2ab830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:00.934319 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d7cbb-b6ed-4290-b654-cc6bca2ab830 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:00.934609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d7cbb-b6ed-4290-b654-cc6bca2ab830 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:00.934887 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d7cbb-b6ed-4290-b654-cc6bca2ab830 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:00.942576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d7cbb-b6ed-4290-b654-cc6bca2ab830 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:00.944024 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3d7cbb-b6ed-4290-b654-cc6bca2ab830 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:00.956675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-6f3d7cbb-b6ed-4290-b654-cc6bca2ab830 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:00.956675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:00.956675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:00.956675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:00.956675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:00.956675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:00.956675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:00.956675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:00.956675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:00.956675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:00.956675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:00.956675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:00.956675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:56:00.956675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:56:00.956675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:56:00.957566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:56:00.957566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:00.957566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:00.957566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:00.957566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:00.957566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:00.957566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:00.957566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:00.957566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:00.957566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:00.957566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:00.957566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:00.957566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:00.957566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:00.957566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:00.957566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:00.957566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:00.958417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:56:00.958417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:56:00.958417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:56:00.958417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:56:00.958417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:56:00.958417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:56:00.958417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:00.958417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:00.961235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 239/487] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:56:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:00.963957 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d1f50-8467-4df4-88c7-4e094eb01b98 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:01.072876 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d1f50-8467-4df4-88c7-4e094eb01b98 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:01.097582 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:01.114152 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8cab2-1f80-46be-b6cb-2c4f5765f507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:01.114777 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8cab2-1f80-46be-b6cb-2c4f5765f507 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:01.115064 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8cab2-1f80-46be-b6cb-2c4f5765f507 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:01.115344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8cab2-1f80-46be-b6cb-2c4f5765f507 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:01.116042 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8cab2-1f80-46be-b6cb-2c4f5765f507 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:01.116356 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8cab2-1f80-46be-b6cb-2c4f5765f507 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:01.117876 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8cab2-1f80-46be-b6cb-2c4f5765f507 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:01.138213 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 249/488] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Oct 22 23:56:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:01.144469 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d7cbb-b6ed-4290-b654-cc6bca2ab830 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:01.233158 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:01.244649 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8cab2-1f80-46be-b6cb-2c4f5765f507 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:01.316951 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d7cbb-b6ed-4290-b654-cc6bca2ab830 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:01.363397 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9299a-d726-4de9-b281-f72ae6ee1c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:01.364390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9299a-d726-4de9-b281-f72ae6ee1c2b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:01.364910 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9299a-d726-4de9-b281-f72ae6ee1c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:01.365380 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9299a-d726-4de9-b281-f72ae6ee1c2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:01.366376 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9299a-d726-4de9-b281-f72ae6ee1c2b None admin] RBAC: Adding query filter params (role.id=73721a054c3842bba868ab52c1a3f8e0, scope.project.id=5c9246c059754504afee4711d113a3fe, user.id=a09d45d080bc461c88f38ff281b75b06) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:01.366878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9299a-d726-4de9-b281-f72ae6ee1c2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:01.368525 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9299a-d726-4de9-b281-f72ae6ee1c2b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:01.375491 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 240/489] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:56:01 2022] GET /identity/v3/role_assignments?user.id=a09d45d080bc461c88f38ff281b75b06&scope.project.id=5c9246c059754504afee4711d113a3fe&role.id=73721a054c3842bba868ab52c1a3f8e0 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:01.377300 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67384942-d98c-40fd-8e74-9c6f67e58e6a None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:01.377796 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67384942-d98c-40fd-8e74-9c6f67e58e6a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:01.378270 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67384942-d98c-40fd-8e74-9c6f67e58e6a None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:01.384041 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8cab2-1f80-46be-b6cb-2c4f5765f507 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:01.384406 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 241/490] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:01.386083 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a29e02-7ff8-461f-bae9-af06e203f8f3 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:01.386648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a29e02-7ff8-461f-bae9-af06e203f8f3 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:01.387132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a29e02-7ff8-461f-bae9-af06e203f8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:01.407512 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-41a29e02-7ff8-461f-bae9-af06e203f8f3 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:01.409095 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-41a29e02-7ff8-461f-bae9-af06e203f8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:01.428825 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4754e-764e-4913-8f2e-5e7328381911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:01.429927 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4754e-764e-4913-8f2e-5e7328381911 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:01.430220 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4754e-764e-4913-8f2e-5e7328381911 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:01.430505 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4754e-764e-4913-8f2e-5e7328381911 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:01.446883 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4754e-764e-4913-8f2e-5e7328381911 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:56:01.453858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-d6a4754e-764e-4913-8f2e-5e7328381911 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:56:01.453858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:01.453858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:01.453858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:01.453858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:01.453858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:01.453858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:01.453858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:01.453858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:01.453858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:01.453858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:01.453858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:01.453858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:56:01.453858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:56:01.453858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:56:01.453858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:56:01.455353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:56:01.455353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:56:01.455353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:56:01.455353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:56:01.455353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:01.455353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:01.455353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:01.455353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:01.455353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:01.455353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:01.455353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:01.455353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:01.455353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:01.455353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:01.455353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:01.455353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:01.456131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:01.456131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:01.456131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:01.456131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:01.456131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:56:01.456131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:56:01.456131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:56:01.456131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:56:01.456131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:56:01.456131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:56:01.456131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:56:01.456131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:01.456753 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 250/491] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Oct 22 23:56:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:01.461618 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4754e-764e-4913-8f2e-5e7328381911 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:01.500518 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 242/492] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:56:01.518522 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:01.593034 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4754e-764e-4913-8f2e-5e7328381911 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:01.647552 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156a584-7df3-46ef-b1b2-1be337f8de5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:01.648505 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156a584-7df3-46ef-b1b2-1be337f8de5a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:01.649008 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156a584-7df3-46ef-b1b2-1be337f8de5a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:01.649494 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156a584-7df3-46ef-b1b2-1be337f8de5a None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:01.650355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156a584-7df3-46ef-b1b2-1be337f8de5a None admin] RBAC: Adding query filter params (name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:01.650977 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156a584-7df3-46ef-b1b2-1be337f8de5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:01.652592 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8156a584-7df3-46ef-b1b2-1be337f8de5a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:01.665302 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 251/493] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Oct 22 23:56:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:01.671396 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a29e02-7ff8-461f-bae9-af06e203f8f3 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:01.723776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:01.789541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a29e02-7ff8-461f-bae9-af06e203f8f3 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:01.827331 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c79808-0997-44a9-8d23-2cf2c59a75d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:01.828301 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c79808-0997-44a9-8d23-2cf2c59a75d9 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:01.828830 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c79808-0997-44a9-8d23-2cf2c59a75d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:01.829740 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c79808-0997-44a9-8d23-2cf2c59a75d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:01.843968 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c79808-0997-44a9-8d23-2cf2c59a75d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:01.846170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c79808-0997-44a9-8d23-2cf2c59a75d9 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:01.863632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-e4c79808-0997-44a9-8d23-2cf2c59a75d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:01.863632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:01.863632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:01.863632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:01.863632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:01.863632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:01.863632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:01.863632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:01.863632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:01.863632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:01.863632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:01.863632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:01.863632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:01.863632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:01.863632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:01.863632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:01.865089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:01.865089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:01.865089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:01.865089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:01.865089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:01.865089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:01.865089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:01.865089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:01.865089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:01.865089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:01.865089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:01.865089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:01.865089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:01.865089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:01.865089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:01.865089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:01.865089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:01.866113 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:01.866113 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:01.866113 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:01.866113 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 243/494] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:01.871610 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156a584-7df3-46ef-b1b2-1be337f8de5a None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:01.902256 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c79808-0997-44a9-8d23-2cf2c59a75d9 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:02.034423 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156a584-7df3-46ef-b1b2-1be337f8de5a None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:02.056860 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:02.067737 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:02.074978 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c79808-0997-44a9-8d23-2cf2c59a75d9 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:02.127745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ef15f-7a92-45ea-a3f7-752c627eaa8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:02.128355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ef15f-7a92-45ea-a3f7-752c627eaa8a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:02.128669 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ef15f-7a92-45ea-a3f7-752c627eaa8a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:02.128946 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ef15f-7a92-45ea-a3f7-752c627eaa8a None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:02.129591 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ef15f-7a92-45ea-a3f7-752c627eaa8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:02.129894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ef15f-7a92-45ea-a3f7-752c627eaa8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:02.131247 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ef15f-7a92-45ea-a3f7-752c627eaa8a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:02.138603 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82c410-a8da-4b7f-9f67-d78b5b52f6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:02.139187 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82c410-a8da-4b7f-9f67-d78b5b52f6e3 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:02.139470 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82c410-a8da-4b7f-9f67-d78b5b52f6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:02.139747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82c410-a8da-4b7f-9f67-d78b5b52f6e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:02.158420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 252/495] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:02.163894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ef15f-7a92-45ea-a3f7-752c627eaa8a None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:02.167322 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f82c410-a8da-4b7f-9f67-d78b5b52f6e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:56:02.176171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-4f82c410-a8da-4b7f-9f67-d78b5b52f6e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:56:02.176171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:02.176171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:02.176171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:02.176171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:02.176171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:02.176171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:02.176171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:02.176171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:02.176171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:02.176171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:02.176171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:02.176171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:56:02.176171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:56:02.176171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:56:02.176171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:56:02.177144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:56:02.177144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:56:02.177144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:56:02.177144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:56:02.177144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:02.177144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:02.177144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:02.177144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:02.177144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:02.177144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:02.177144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:02.177144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:02.177144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:02.177144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:02.177144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:02.177144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:02.178076 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:02.178076 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:02.178076 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:02.178076 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:02.178076 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:56:02.178076 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:56:02.178076 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:56:02.178076 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:56:02.178076 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:56:02.178076 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:56:02.178076 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:56:02.178076 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:02.181094 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 244/496] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Oct 22 23:56:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:56:02.187680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82c410-a8da-4b7f-9f67-d78b5b52f6e3 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:02.302079 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82c410-a8da-4b7f-9f67-d78b5b52f6e3 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:02.336542 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e2084-d01f-4805-9474-6b6c7741ed16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:02.337160 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e2084-d01f-4805-9474-6b6c7741ed16 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:02.337443 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e2084-d01f-4805-9474-6b6c7741ed16 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:02.337720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e2084-d01f-4805-9474-6b6c7741ed16 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:02.338387 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e2084-d01f-4805-9474-6b6c7741ed16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:02.338764 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e2084-d01f-4805-9474-6b6c7741ed16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:02.340250 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e2084-d01f-4805-9474-6b6c7741ed16 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:02.350665 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 245/497] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:56:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:02.356287 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e2084-d01f-4805-9474-6b6c7741ed16 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:02.375817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ef15f-7a92-45ea-a3f7-752c627eaa8a None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:02.377271 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:02.467300 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e2084-d01f-4805-9474-6b6c7741ed16 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:02.498129 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf33e0-7a38-4770-b0da-ab44d115b742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:02.498779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf33e0-7a38-4770-b0da-ab44d115b742 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:02.499064 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf33e0-7a38-4770-b0da-ab44d115b742 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:02.499386 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf33e0-7a38-4770-b0da-ab44d115b742 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:02.511181 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b781312-d2be-48b9-b426-93b5aa9f1c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:02.511992 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b781312-d2be-48b9-b426-93b5aa9f1c4f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:02.512282 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b781312-d2be-48b9-b426-93b5aa9f1c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:02.512581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b781312-d2be-48b9-b426-93b5aa9f1c4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:02.521413 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf33e0-7a38-4770-b0da-ab44d115b742 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:02.522911 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf33e0-7a38-4770-b0da-ab44d115b742 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:02.528478 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b781312-d2be-48b9-b426-93b5aa9f1c4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:02.529970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b781312-d2be-48b9-b426-93b5aa9f1c4f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:02.546025 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-afdf33e0-7a38-4770-b0da-ab44d115b742 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 22 23:56:02.546025 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:02.546025 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:02.546025 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:02.546025 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:02.546025 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:02.546025 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:02.546025 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:02.546025 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:02.546025 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:02.546025 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:02.546025 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:02.546025 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:56:02.546025 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:56:02.546025 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:56:02.546025 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:56:02.547074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:02.547074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:02.547074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:56:02.547074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:02.547074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:56:02.547074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:02.547074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:02.547074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:02.547074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:02.547074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:02.547074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:02.547074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:02.547074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:02.547074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:02.547074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:02.547074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:02.547074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:02.547074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:02.548706 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:02.548706 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:02.548706 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:56:02.548706 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:56:02.548706 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:56:02.548706 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:56:02.548706 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:56:02.548706 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:56:02.548706 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 22 23:56:02.548706 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:02.549235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-6b781312-d2be-48b9-b426-93b5aa9f1c4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:02.549235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:02.549235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:02.549235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:02.549235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:02.549235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:02.549235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:02.549235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:02.549235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:02.549235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:02.549235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:02.549235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:02.549235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:02.549235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:02.549235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:02.549235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:02.550119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:02.550119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:02.550119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:02.550119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:02.550119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:02.550119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:02.550119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:02.550119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:02.550119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:02.550119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:02.550119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:02.550119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:02.550119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:02.550119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:02.550119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:02.550119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:02.550119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:02.551012 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:02.551012 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:02.551012 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:02.551674 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 253/498] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Oct 22 23:56:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:02.556816 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 246/499] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:02.558406 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf33e0-7a38-4770-b0da-ab44d115b742 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:02.558613 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b781312-d2be-48b9-b426-93b5aa9f1c4f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:02.605243 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:02.632881 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:02.669796 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b781312-d2be-48b9-b426-93b5aa9f1c4f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:02.708628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448263c-7f9c-4cb6-834f-a745ce982091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:02.709617 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448263c-7f9c-4cb6-834f-a745ce982091 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:02.710131 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448263c-7f9c-4cb6-834f-a745ce982091 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:02.710611 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448263c-7f9c-4cb6-834f-a745ce982091 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:02.711559 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448263c-7f9c-4cb6-834f-a745ce982091 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:02.712065 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448263c-7f9c-4cb6-834f-a745ce982091 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:02.713666 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448263c-7f9c-4cb6-834f-a745ce982091 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:02.726884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 247/500] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:02.732580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448263c-7f9c-4cb6-834f-a745ce982091 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:02.739126 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf33e0-7a38-4770-b0da-ab44d115b742 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:02.814210 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb2f82-2e59-442b-8471-2354eada079f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:02.815315 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb2f82-2e59-442b-8471-2354eada079f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:02.815839 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb2f82-2e59-442b-8471-2354eada079f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:02.816321 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb2f82-2e59-442b-8471-2354eada079f None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:02.817580 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb2f82-2e59-442b-8471-2354eada079f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:02.818113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb2f82-2e59-442b-8471-2354eada079f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:02.819798 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb2f82-2e59-442b-8471-2354eada079f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:02.857192 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 254/501] 10.222.0.13 () {56 vars in 1135 bytes} [Sat Oct 22 23:56:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:02.863996 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb2f82-2e59-442b-8471-2354eada079f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:02.939884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448263c-7f9c-4cb6-834f-a745ce982091 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:02.996780 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:03.011250 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee13921-75b1-49a2-bf8e-562cd176deae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:03.011885 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:03.012347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee13921-75b1-49a2-bf8e-562cd176deae None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:03.012599 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee13921-75b1-49a2-bf8e-562cd176deae None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:03.012878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee13921-75b1-49a2-bf8e-562cd176deae None admin] PATH_INFO: `/v3/users/nova` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:03.027474 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee13921-75b1-49a2-bf8e-562cd176deae None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:03.029016 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee13921-75b1-49a2-bf8e-562cd176deae None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:03.050006 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb2f82-2e59-442b-8471-2354eada079f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:03.055629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-5ee13921-75b1-49a2-bf8e-562cd176deae None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 22 23:56:03.055629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:03.055629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:03.055629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:03.055629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:03.055629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:03.055629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:03.055629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:03.055629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:03.055629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:03.055629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:03.055629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:03.055629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:56:03.055629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:56:03.055629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:56:03.055629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:56:03.056792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:03.056792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:03.056792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:56:03.056792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:03.056792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:56:03.056792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:03.056792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:03.056792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:03.056792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:03.056792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:03.056792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:03.056792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:03.056792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:03.056792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:03.056792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:03.056792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:03.056792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:03.056792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:03.057750 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:03.057750 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:03.057750 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:56:03.057750 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:56:03.057750 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:56:03.057750 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:56:03.057750 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:56:03.057750 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:56:03.057750 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 22 23:56:03.057750 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:03.057750 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 248/502] 10.222.0.13 () {56 vars in 1093 bytes} [Sat Oct 22 23:56:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:03.063178 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee13921-75b1-49a2-bf8e-562cd176deae None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:03.145133 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcfc7cd-fe62-41a0-870a-73757a7628e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:03.145748 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfc7cd-fe62-41a0-870a-73757a7628e4 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:03.146034 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfc7cd-fe62-41a0-870a-73757a7628e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:03.146304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfc7cd-fe62-41a0-870a-73757a7628e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:03.212717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee13921-75b1-49a2-bf8e-562cd176deae None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:03.233705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcfc7cd-fe62-41a0-870a-73757a7628e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:03.235148 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcfc7cd-fe62-41a0-870a-73757a7628e4 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:03.257463 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3c6f33-b068-4dcc-9e84-caf4ece392d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:03.258509 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c6f33-b068-4dcc-9e84-caf4ece392d6 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:03.258799 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c6f33-b068-4dcc-9e84-caf4ece392d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:03.259334 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3c6f33-b068-4dcc-9e84-caf4ece392d6 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:03.260086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c6f33-b068-4dcc-9e84-caf4ece392d6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:03.260676 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c6f33-b068-4dcc-9e84-caf4ece392d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:03.262856 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3c6f33-b068-4dcc-9e84-caf4ece392d6 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:03.283964 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 249/503] 10.222.0.13 () {56 vars in 1133 bytes} [Sat Oct 22 23:56:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:03.285268 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6654a-3b00-423e-8d78-7afb09d4f6c0 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:03.285538 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6654a-3b00-423e-8d78-7afb09d4f6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:03.285826 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6654a-3b00-423e-8d78-7afb09d4f6c0 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:03.287276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 250/504] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:03.288306 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5934b2d-765e-4030-ab84-2818e6c631c3 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:03.288613 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5934b2d-765e-4030-ab84-2818e6c631c3 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:03.288890 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5934b2d-765e-4030-ab84-2818e6c631c3 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:03.290191 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 251/505] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:03.293260 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5934b2d-765e-4030-ab84-2818e6c631c3 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:03.326168 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:03.346592 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 255/506] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:56:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 22 23:56:03.346926 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:03.348126 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de194aa-ccbe-4e8c-8d49-4aded6439193 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:03.348429 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de194aa-ccbe-4e8c-8d49-4aded6439193 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:03.348741 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de194aa-ccbe-4e8c-8d49-4aded6439193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:03.367188 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-0de194aa-ccbe-4e8c-8d49-4aded6439193 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:03.368390 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-0de194aa-ccbe-4e8c-8d49-4aded6439193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:03.409894 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5934b2d-765e-4030-ab84-2818e6c631c3 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:03.435360 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 256/507] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:56:03.436616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab452f-7c3f-4f1f-a9a8-e5b5a9ebf47a None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:03.436910 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab452f-7c3f-4f1f-a9a8-e5b5a9ebf47a None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:03.437192 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab452f-7c3f-4f1f-a9a8-e5b5a9ebf47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:03.453873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e1456-9974-4a98-b410-44c18e90f748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:03.455406 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e1456-9974-4a98-b410-44c18e90f748 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:03.455942 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e1456-9974-4a98-b410-44c18e90f748 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:03.456049 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-7eab452f-7c3f-4f1f-a9a8-e5b5a9ebf47a None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:03.456552 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e1456-9974-4a98-b410-44c18e90f748 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:03.457256 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-7eab452f-7c3f-4f1f-a9a8-e5b5a9ebf47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:03.467723 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e1456-9974-4a98-b410-44c18e90f748 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:03.469556 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e1456-9974-4a98-b410-44c18e90f748 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:03.486275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-3c9e1456-9974-4a98-b410-44c18e90f748 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:03.486275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:03.486275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:03.486275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:03.486275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:03.486275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:03.486275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:03.486275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:03.486275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:03.486275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:03.486275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:03.486275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:03.486275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:03.486275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:03.486275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:03.486275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:03.487236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:03.487236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:03.487236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:03.487236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:03.487236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:03.487236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:03.487236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:03.487236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:03.487236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:03.487236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:03.487236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:03.487236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:03.487236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:03.487236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:03.487236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:03.487236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:03.487236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:03.488138 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:03.488138 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:03.488138 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:03.490295 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 252/508] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:03.491318 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e1456-9974-4a98-b410-44c18e90f748 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:03.548713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 257/509] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:56:03.549844 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab452f-7c3f-4f1f-a9a8-e5b5a9ebf47a None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:03.635340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:03.656642 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:03.666860 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e1456-9974-4a98-b410-44c18e90f748 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:03.699287 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab452f-7c3f-4f1f-a9a8-e5b5a9ebf47a None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:03.739000 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c742f9a-589a-4b64-85d9-3cfbe61cb11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:03.739907 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c742f9a-589a-4b64-85d9-3cfbe61cb11e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:03.741241 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c742f9a-589a-4b64-85d9-3cfbe61cb11e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:03.742161 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c742f9a-589a-4b64-85d9-3cfbe61cb11e None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:03.743013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c742f9a-589a-4b64-85d9-3cfbe61cb11e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:03.743964 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c742f9a-589a-4b64-85d9-3cfbe61cb11e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:03.746873 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c742f9a-589a-4b64-85d9-3cfbe61cb11e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:03.747715 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6071eb4-fb81-435a-bddb-79287345eb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:03.749882 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6071eb4-fb81-435a-bddb-79287345eb08 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:03.750370 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6071eb4-fb81-435a-bddb-79287345eb08 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:03.750826 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6071eb4-fb81-435a-bddb-79287345eb08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:03.769930 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 258/510] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:03.776571 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6071eb4-fb81-435a-bddb-79287345eb08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:03.776571 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6071eb4-fb81-435a-bddb-79287345eb08 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:03.779450 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c742f9a-589a-4b64-85d9-3cfbe61cb11e None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:03.793645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-e6071eb4-fb81-435a-bddb-79287345eb08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:03.793645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:03.793645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:03.793645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:03.793645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:03.793645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:03.793645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:03.793645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:03.793645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:03.793645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:03.793645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:03.793645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:03.793645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:03.793645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:03.793645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:03.793645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:03.794873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:03.794873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:03.794873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:03.794873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:03.794873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:03.794873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:03.794873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:03.794873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:03.794873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:03.794873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:03.794873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:03.794873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:03.794873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:03.794873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:03.794873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:03.794873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:03.794873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:03.795787 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:03.795787 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:03.795787 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:03.795787 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 253/511] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:03.800693 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6071eb4-fb81-435a-bddb-79287345eb08 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:03.910811 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6071eb4-fb81-435a-bddb-79287345eb08 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:03.911021 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c742f9a-589a-4b64-85d9-3cfbe61cb11e None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:03.912425 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:03.950291 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9c272-dfa4-49ac-9cce-d9bcbdce5d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:03.951024 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9c272-dfa4-49ac-9cce-d9bcbdce5d56 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:03.951307 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9c272-dfa4-49ac-9cce-d9bcbdce5d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:03.951586 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9c272-dfa4-49ac-9cce-d9bcbdce5d56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:03.955895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb30aa-1f8e-44c1-a257-582629477ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:03.956559 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb30aa-1f8e-44c1-a257-582629477ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:03.956849 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb30aa-1f8e-44c1-a257-582629477ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:03.957127 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb30aa-1f8e-44c1-a257-582629477ff7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:03.962551 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9c272-dfa4-49ac-9cce-d9bcbdce5d56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:03.963998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9c272-dfa4-49ac-9cce-d9bcbdce5d56 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:03.964122 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb30aa-1f8e-44c1-a257-582629477ff7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:03.965560 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb30aa-1f8e-44c1-a257-582629477ff7 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:03.977537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-faeb30aa-1f8e-44c1-a257-582629477ff7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:03.977537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:03.977537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:03.977537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:03.977537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:03.977537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:03.977537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:03.977537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:03.977537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:03.977537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:03.977537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:03.977537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:03.977537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:56:03.977537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:56:03.977537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:56:03.978454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:56:03.978454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:03.978454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:03.978454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:03.978454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:03.978454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:03.978454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:03.978454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:03.978454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:03.978454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:03.978454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:03.978454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:03.978454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:03.978454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:03.978454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:03.978454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:03.978454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:03.979375 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:56:03.979375 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:56:03.979375 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:56:03.979375 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:56:03.979375 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:56:03.979375 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:56:03.979375 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:03.979375 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:03.980314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-45a9c272-dfa4-49ac-9cce-d9bcbdce5d56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:03.980314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:03.980314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:03.980314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:03.980314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:03.980314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:03.980314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:03.980314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:03.980314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:03.980314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:03.980314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:03.980314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:03.980314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:03.980314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:03.980314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:03.980314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:03.986989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:03.986989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:03.986989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:03.986989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:03.986989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:03.986989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:03.986989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:03.986989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:03.986989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:03.986989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:03.986989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:03.986989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:03.986989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:03.986989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:03.986989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:03.986989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:03.986989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:03.987894 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:03.987894 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:03.987894 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:03.987894 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 254/512] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:56:03.987894 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9c272-dfa4-49ac-9cce-d9bcbdce5d56 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:03.988196 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 259/513] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:56:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:56:03.988196 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb30aa-1f8e-44c1-a257-582629477ff7 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:03.988435 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:04.099327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9c272-dfa4-49ac-9cce-d9bcbdce5d56 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:04.121266 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb30aa-1f8e-44c1-a257-582629477ff7 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:04.134249 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938dbbb6-c994-45ad-bda3-91c261988a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:04.134859 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dbbb6-c994-45ad-bda3-91c261988a5b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:04.135136 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dbbb6-c994-45ad-bda3-91c261988a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:04.135408 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dbbb6-c994-45ad-bda3-91c261988a5b None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:04.136046 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938dbbb6-c994-45ad-bda3-91c261988a5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:04.136348 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938dbbb6-c994-45ad-bda3-91c261988a5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:04.137783 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938dbbb6-c994-45ad-bda3-91c261988a5b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:04.149656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 255/514] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:04.154931 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938dbbb6-c994-45ad-bda3-91c261988a5b None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:04.161416 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b9b7c-2494-456a-845c-f8bec46d3341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:04.162052 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b9b7c-2494-456a-845c-f8bec46d3341 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:04.162340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b9b7c-2494-456a-845c-f8bec46d3341 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:04.162620 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b9b7c-2494-456a-845c-f8bec46d3341 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:04.163204 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b9b7c-2494-456a-845c-f8bec46d3341 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:04.164599 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b9b7c-2494-456a-845c-f8bec46d3341 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:04.176143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-f66b9b7c-2494-456a-845c-f8bec46d3341 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 22 23:56:04.176143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:04.176143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:56:04.176143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:56:04.176143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:56:04.176143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:56:04.176143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:56:04.176143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:04.176143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 22 23:56:04.176143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:04.176143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:04.176143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:04.176143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:04.176143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:04.176143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:04.176143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:04.176143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:04.177358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:04.177358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:04.177358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:04.177358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:04.177358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:04.177358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:04.177358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:56:04.177358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:56:04.177358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:56:04.177358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:56:04.177358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:04.177358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:04.177358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:04.177358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:04.177358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:04.177358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:04.178437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:04.178437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:04.178437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:04.178437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:04.178437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:04.178437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:04.178437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:04.178437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:04.178437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:04.178437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:04.178437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:56:04.178437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:04.178437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 22 23:56:04.178437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:04.178437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 260/515] 10.222.0.13 () {56 vars in 1129 bytes} [Sat Oct 22 23:56:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:04.179407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b9b7c-2494-456a-845c-f8bec46d3341 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:04.206687 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:04.219226 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:04.257318 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938dbbb6-c994-45ad-bda3-91c261988a5b None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:04.278926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b9b7c-2494-456a-845c-f8bec46d3341 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:04.290812 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ffea5f-4964-421b-86e8-62436bebb07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:04.291406 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffea5f-4964-421b-86e8-62436bebb07e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:04.291685 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffea5f-4964-421b-86e8-62436bebb07e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:04.291959 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffea5f-4964-421b-86e8-62436bebb07e None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:04.292674 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ffea5f-4964-421b-86e8-62436bebb07e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:04.292978 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ffea5f-4964-421b-86e8-62436bebb07e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:04.294393 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ffea5f-4964-421b-86e8-62436bebb07e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:04.305675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 256/516] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Oct 22 23:56:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:04.306467 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ffea5f-4964-421b-86e8-62436bebb07e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:04.318002 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2690738-532d-495c-afc1-102ca7153854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:04.318614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2690738-532d-495c-afc1-102ca7153854 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:04.318900 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2690738-532d-495c-afc1-102ca7153854 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:04.319180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2690738-532d-495c-afc1-102ca7153854 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:04.319823 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2690738-532d-495c-afc1-102ca7153854 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:04.320132 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2690738-532d-495c-afc1-102ca7153854 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:04.321533 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2690738-532d-495c-afc1-102ca7153854 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:04.333190 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 261/517] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:04.334341 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68656442-523b-4a45-86e8-b05436e25bea None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:04.334630 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68656442-523b-4a45-86e8-b05436e25bea None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:04.334982 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68656442-523b-4a45-86e8-b05436e25bea None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:04.336243 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 262/518] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:04.336813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68656442-523b-4a45-86e8-b05436e25bea None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:04.410930 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ffea5f-4964-421b-86e8-62436bebb07e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:04.429281 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:04.433142 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:04.446157 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe27a9-f667-47fc-af40-4aadd5c41b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:04.446789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe27a9-f667-47fc-af40-4aadd5c41b15 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:04.447037 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68656442-523b-4a45-86e8-b05436e25bea None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:04.447171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe27a9-f667-47fc-af40-4aadd5c41b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:04.447324 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe27a9-f667-47fc-af40-4aadd5c41b15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:04.454855 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe27a9-f667-47fc-af40-4aadd5c41b15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:04.456774 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe27a9-f667-47fc-af40-4aadd5c41b15 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:04.467939 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-7fbe27a9-f667-47fc-af40-4aadd5c41b15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:04.467939 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:04.467939 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:04.467939 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:04.467939 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:04.467939 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:04.467939 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:04.467939 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:04.467939 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:04.467939 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:04.467939 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:04.467939 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:04.467939 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:56:04.467939 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:56:04.467939 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:56:04.468883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:56:04.468883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:04.468883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:04.468883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:04.468883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:04.468883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:04.468883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:04.468883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:04.468883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:04.468883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:04.468883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:04.468883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:04.468883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:04.468883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:04.468883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:04.468883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:04.468883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:04.470435 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:56:04.470435 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:56:04.470435 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:56:04.470435 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:56:04.470435 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:56:04.470435 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:56:04.470435 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:04.470435 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:04.470435 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 257/519] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:56:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:04.471467 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe27a9-f667-47fc-af40-4aadd5c41b15 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:04.483353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a0188-6fec-42b3-9f6e-922cbfd0875d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:04.483957 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a0188-6fec-42b3-9f6e-922cbfd0875d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:04.484244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a0188-6fec-42b3-9f6e-922cbfd0875d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:04.484541 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a0188-6fec-42b3-9f6e-922cbfd0875d None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:04.485167 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a0188-6fec-42b3-9f6e-922cbfd0875d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:04.485595 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a0188-6fec-42b3-9f6e-922cbfd0875d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:04.486927 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a0188-6fec-42b3-9f6e-922cbfd0875d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:04.491292 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 263/520] 10.222.0.13 () {56 vars in 1124 bytes} [Sat Oct 22 23:56:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:04.492038 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a0188-6fec-42b3-9f6e-922cbfd0875d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:04.569697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe27a9-f667-47fc-af40-4aadd5c41b15 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:04.592283 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a0188-6fec-42b3-9f6e-922cbfd0875d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:04.602429 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c9e79-133e-46fe-9687-15095ae871c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:04.603087 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c9e79-133e-46fe-9687-15095ae871c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:04.603370 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c9e79-133e-46fe-9687-15095ae871c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:04.603647 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c9e79-133e-46fe-9687-15095ae871c8 None admin] PATH_INFO: `/v3/projects/5c9246c059754504afee4711d113a3fe/users/c4a3579d56294b9d8ad36d4d174c8526/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:04.605570 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c9e79-133e-46fe-9687-15095ae871c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c9246c059754504afee4711d113a3fe, user_id=c4a3579d56294b9d8ad36d4d174c8526, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:04.607012 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c9e79-133e-46fe-9687-15095ae871c8 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:04.620059 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 258/521] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:56:04 2022] PUT /identity/v3/projects/5c9246c059754504afee4711d113a3fe/users/c4a3579d56294b9d8ad36d4d174c8526/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:56:04.621254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c048a75-3419-475a-92a4-a6e2f90a6fe9 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:04.621536 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c048a75-3419-475a-92a4-a6e2f90a6fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:04.621811 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c048a75-3419-475a-92a4-a6e2f90a6fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:04.637967 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51317f-4418-4a16-a53b-2bfe1387cc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:04.638592 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51317f-4418-4a16-a53b-2bfe1387cc71 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:04.638873 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51317f-4418-4a16-a53b-2bfe1387cc71 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:04.639147 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51317f-4418-4a16-a53b-2bfe1387cc71 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:04.639807 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51317f-4418-4a16-a53b-2bfe1387cc71 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:04.641304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51317f-4418-4a16-a53b-2bfe1387cc71 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:04.641794 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-2c048a75-3419-475a-92a4-a6e2f90a6fe9 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:04.643047 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2c048a75-3419-475a-92a4-a6e2f90a6fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:04.664312 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 264/522] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Oct 22 23:56:04 2022] POST /identity/v3/users => generated 254 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:56:04.671458 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51317f-4418-4a16-a53b-2bfe1387cc71 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:04.688247 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:04.698023 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:04.747332 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 259/523] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:56:04.748387 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c048a75-3419-475a-92a4-a6e2f90a6fe9 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:04.851520 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51317f-4418-4a16-a53b-2bfe1387cc71 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:04.900263 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c048a75-3419-475a-92a4-a6e2f90a6fe9 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:04.903682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17480bf-6cec-42f7-a930-bd18cd039cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:04.904293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17480bf-6cec-42f7-a930-bd18cd039cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:04.904596 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17480bf-6cec-42f7-a930-bd18cd039cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:04.904871 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17480bf-6cec-42f7-a930-bd18cd039cc9 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:04.905560 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17480bf-6cec-42f7-a930-bd18cd039cc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:04.905865 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17480bf-6cec-42f7-a930-bd18cd039cc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:04.909013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17480bf-6cec-42f7-a930-bd18cd039cc9 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:04.924781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 265/524] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Oct 22 23:56:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:04.930380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17480bf-6cec-42f7-a930-bd18cd039cc9 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:04.951464 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d72f47-e7fe-44bd-a606-992f4663d22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:04.952178 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d72f47-e7fe-44bd-a606-992f4663d22c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:04.952579 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d72f47-e7fe-44bd-a606-992f4663d22c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:04.952986 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d72f47-e7fe-44bd-a606-992f4663d22c None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:04.953728 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d72f47-e7fe-44bd-a606-992f4663d22c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:04.954655 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d72f47-e7fe-44bd-a606-992f4663d22c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:04.956115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d72f47-e7fe-44bd-a606-992f4663d22c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:04.968513 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 260/525] 10.222.0.13 () {56 vars in 1124 bytes} [Sat Oct 22 23:56:04 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:05.044956 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:05.062611 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d72f47-e7fe-44bd-a606-992f4663d22c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:05.108765 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:05.146290 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17480bf-6cec-42f7-a930-bd18cd039cc9 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:05.213202 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf73770-861a-4eed-ac58-9fba8b2bcce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:05.213739 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf73770-861a-4eed-ac58-9fba8b2bcce0 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:05.214024 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf73770-861a-4eed-ac58-9fba8b2bcce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:05.214309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf73770-861a-4eed-ac58-9fba8b2bcce0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:05.215039 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf73770-861a-4eed-ac58-9fba8b2bcce0 None admin] RBAC: Adding query filter params (role.id=73721a054c3842bba868ab52c1a3f8e0, scope.project.id=5c9246c059754504afee4711d113a3fe, user.id=bfd8c7516b7c435d88298672b2596d6d) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:05.215355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf73770-861a-4eed-ac58-9fba8b2bcce0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:05.216814 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf73770-861a-4eed-ac58-9fba8b2bcce0 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:05.224960 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 266/526] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:56:04 2022] GET /identity/v3/role_assignments?user.id=bfd8c7516b7c435d88298672b2596d6d&scope.project.id=5c9246c059754504afee4711d113a3fe&role.id=73721a054c3842bba868ab52c1a3f8e0 => generated 608 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:05.228528 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d72f47-e7fe-44bd-a606-992f4663d22c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:05.272628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc29dc9-2394-48e3-8ab6-44ab056927bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:05.273289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc29dc9-2394-48e3-8ab6-44ab056927bd None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:05.273567 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc29dc9-2394-48e3-8ab6-44ab056927bd None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:05.273837 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc29dc9-2394-48e3-8ab6-44ab056927bd None admin] PATH_INFO: `/v3/services/network` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:05.274430 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc29dc9-2394-48e3-8ab6-44ab056927bd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:05.275806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc29dc9-2394-48e3-8ab6-44ab056927bd None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:05.290970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-3bc29dc9-2394-48e3-8ab6-44ab056927bd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 22 23:56:05.290970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:05.290970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:56:05.290970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:56:05.290970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:56:05.290970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:56:05.290970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:56:05.290970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:05.290970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 22 23:56:05.290970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:05.290970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:05.290970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:05.290970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:05.290970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:05.290970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:05.290970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:05.290970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:05.292071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:05.292071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:05.292071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:05.292071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:05.292071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:05.292071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:05.292071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:56:05.292071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:56:05.292071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:56:05.292071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:56:05.292071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:05.292071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:05.292071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:05.292071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:05.292071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:05.292071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:05.293595 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:05.293595 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:05.293595 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:05.293595 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:05.293595 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:05.293595 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:05.293595 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:05.293595 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:05.293595 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:05.293595 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:05.293595 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:56:05.293595 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:05.293595 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 22 23:56:05.293595 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:05.293595 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 261/527] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:56:05 2022] GET /identity/v3/services/network => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:56:05.298819 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf73770-861a-4eed-ac58-9fba8b2bcce0 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:05.464467 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf73770-861a-4eed-ac58-9fba8b2bcce0 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:05.504848 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:05.511853 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41ac5d-c9fa-4eb3-96b1-1717a4cf57a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:05.513129 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41ac5d-c9fa-4eb3-96b1-1717a4cf57a9 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:05.513416 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41ac5d-c9fa-4eb3-96b1-1717a4cf57a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:05.513801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41ac5d-c9fa-4eb3-96b1-1717a4cf57a9 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:05.514444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41ac5d-c9fa-4eb3-96b1-1717a4cf57a9 None admin] RBAC: Adding query filter params (name=network) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:05.514793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41ac5d-c9fa-4eb3-96b1-1717a4cf57a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:05.517214 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41ac5d-c9fa-4eb3-96b1-1717a4cf57a9 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:05.524102 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 267/528] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Oct 22 23:56:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:05.529418 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc29dc9-2394-48e3-8ab6-44ab056927bd None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:05.635027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:05.639111 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc29dc9-2394-48e3-8ab6-44ab056927bd None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:05.676957 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514b8ed-a24d-417a-8841-375a9204c0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:05.677663 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514b8ed-a24d-417a-8841-375a9204c0db None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:05.677943 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514b8ed-a24d-417a-8841-375a9204c0db None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:05.678216 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514b8ed-a24d-417a-8841-375a9204c0db None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:05.678914 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f514b8ed-a24d-417a-8841-375a9204c0db None admin] RBAC: Adding query filter params (type=network) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:05.679215 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f514b8ed-a24d-417a-8841-375a9204c0db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:05.681038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f514b8ed-a24d-417a-8841-375a9204c0db None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:05.686439 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 262/529] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Oct 22 23:56:05 2022] GET /identity/v3/services?type=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:07.352087 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727a70b-ec7e-41fa-9bc2-6a4971330239 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:07.353121 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727a70b-ec7e-41fa-9bc2-6a4971330239 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:07.353620 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727a70b-ec7e-41fa-9bc2-6a4971330239 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:07.355607 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 268/530] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:07.361578 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb29da5-a86f-498c-8efa-d6f661e82701 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:07.362002 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb29da5-a86f-498c-8efa-d6f661e82701 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:07.362238 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb29da5-a86f-498c-8efa-d6f661e82701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:07.385286 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-beb29da5-a86f-498c-8efa-d6f661e82701 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:07.386601 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-beb29da5-a86f-498c-8efa-d6f661e82701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:07.388696 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d81a28-2d91-4936-8926-a1951443aa0c None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:07.388993 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d81a28-2d91-4936-8926-a1951443aa0c None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:07.389330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d81a28-2d91-4936-8926-a1951443aa0c None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:07.390986 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 269/531] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:07.399553 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24734f7f-a74e-4479-805b-c86e20bb3b24 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:07.399842 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24734f7f-a74e-4479-805b-c86e20bb3b24 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:07.400111 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24734f7f-a74e-4479-805b-c86e20bb3b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:07.429513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-24734f7f-a74e-4479-805b-c86e20bb3b24 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:07.430716 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-24734f7f-a74e-4479-805b-c86e20bb3b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:07.463073 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 263/532] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:56:07.542752 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 270/533] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:56:07.761790 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba76c4-1603-430f-b75c-7e54bc2364a4 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:07.762304 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba76c4-1603-430f-b75c-7e54bc2364a4 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:07.762739 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba76c4-1603-430f-b75c-7e54bc2364a4 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:07.764528 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 264/534] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:07.769724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005e927-f65d-4dc7-9207-9e29164dba40 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:07.770121 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005e927-f65d-4dc7-9207-9e29164dba40 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:07.770494 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005e927-f65d-4dc7-9207-9e29164dba40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:07.796651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-9005e927-f65d-4dc7-9207-9e29164dba40 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:07.797900 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9005e927-f65d-4dc7-9207-9e29164dba40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:07.835149 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba76c4-1603-430f-b75c-7e54bc2364a4 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:07.900480 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 271/535] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:56:07.986793 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:07.994033 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005e927-f65d-4dc7-9207-9e29164dba40 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:07.995933 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba76c4-1603-430f-b75c-7e54bc2364a4 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:08.033931 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ccf413-d593-4591-8c29-f1e2ef9cac70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:08.034822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccf413-d593-4591-8c29-f1e2ef9cac70 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:08.035256 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccf413-d593-4591-8c29-f1e2ef9cac70 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:08.035684 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccf413-d593-4591-8c29-f1e2ef9cac70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:08.050714 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ccf413-d593-4591-8c29-f1e2ef9cac70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:56:08.055999 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-c8ccf413-d593-4591-8c29-f1e2ef9cac70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:56:08.055999 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:08.055999 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:08.055999 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:08.055999 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:08.055999 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:08.055999 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:08.055999 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:08.055999 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:08.055999 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:08.055999 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:08.055999 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:08.055999 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:56:08.055999 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:56:08.055999 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:56:08.055999 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:56:08.057173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:56:08.057173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:56:08.057173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:56:08.057173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:56:08.057173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:08.057173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:08.057173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:08.057173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:08.057173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:08.057173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:08.057173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:08.057173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:08.057173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:08.057173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:08.057173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:08.057173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:08.058235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:08.058235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:08.058235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:08.058235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:08.058235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:56:08.058235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:56:08.058235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:56:08.058235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:56:08.058235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:56:08.058235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:56:08.058235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:56:08.058235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:08.060531 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 265/536] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Oct 22 23:56:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:08.061934 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142dc8f6-28e8-4cd9-a8c5-29ac9365ad0c None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:08.062377 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142dc8f6-28e8-4cd9-a8c5-29ac9365ad0c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:08.062802 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142dc8f6-28e8-4cd9-a8c5-29ac9365ad0c None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:08.064647 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 266/537] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:08.066380 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142dc8f6-28e8-4cd9-a8c5-29ac9365ad0c None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:08.095441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:08.144003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005e927-f65d-4dc7-9207-9e29164dba40 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:08.181116 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb91db0-e20a-41d5-acee-2e34f76506a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:08.181898 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb91db0-e20a-41d5-acee-2e34f76506a0 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:08.182252 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb91db0-e20a-41d5-acee-2e34f76506a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:08.182361 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142dc8f6-28e8-4cd9-a8c5-29ac9365ad0c None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:08.182611 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb91db0-e20a-41d5-acee-2e34f76506a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:08.194361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb91db0-e20a-41d5-acee-2e34f76506a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:56:08.199673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-dcb91db0-e20a-41d5-acee-2e34f76506a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:56:08.199673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:08.199673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:08.199673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:08.199673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:08.199673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:08.199673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:08.199673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:08.199673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:08.199673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:08.199673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:08.199673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:08.199673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:56:08.199673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:56:08.199673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:56:08.199673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:56:08.200651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:56:08.200651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:56:08.200651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:56:08.200651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:56:08.200651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:08.200651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:08.200651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:08.200651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:08.200651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:08.200651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:08.200651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:08.200651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:08.200651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:08.200651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:08.200651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:08.200651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:08.201731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:08.201731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:08.201731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:08.201731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:08.201731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:56:08.201731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:56:08.201731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:56:08.201731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:56:08.201731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:56:08.201731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:56:08.201731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:56:08.201731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:08.201731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 272/538] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Oct 22 23:56:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:56:08.202896 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf00ccd-2de0-44b6-bb28-d7d34ca993be None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:08.203251 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf00ccd-2de0-44b6-bb28-d7d34ca993be None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:08.203598 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf00ccd-2de0-44b6-bb28-d7d34ca993be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:08.223088 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-fbf00ccd-2de0-44b6-bb28-d7d34ca993be None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:08.223400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7a2bc-9905-4be5-8f73-778e617d2691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:08.224068 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7a2bc-9905-4be5-8f73-778e617d2691 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:08.224349 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7a2bc-9905-4be5-8f73-778e617d2691 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:08.224612 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-fbf00ccd-2de0-44b6-bb28-d7d34ca993be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:08.224761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7a2bc-9905-4be5-8f73-778e617d2691 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:08.225380 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7a2bc-9905-4be5-8f73-778e617d2691 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:08.225683 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7a2bc-9905-4be5-8f73-778e617d2691 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:08.227270 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7a2bc-9905-4be5-8f73-778e617d2691 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:08.237164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 267/539] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:56:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:08.237979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7a2bc-9905-4be5-8f73-778e617d2691 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:08.257815 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:08.315874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 273/540] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:56:08.315874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf00ccd-2de0-44b6-bb28-d7d34ca993be None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:08.363574 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:08.385784 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7a2bc-9905-4be5-8f73-778e617d2691 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:08.436877 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf00ccd-2de0-44b6-bb28-d7d34ca993be None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:08.437860 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7731fb8-fd8a-40ed-9e01-6bf61cff9db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:08.438627 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7731fb8-fd8a-40ed-9e01-6bf61cff9db0 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:08.439285 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7731fb8-fd8a-40ed-9e01-6bf61cff9db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:08.439285 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7731fb8-fd8a-40ed-9e01-6bf61cff9db0 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:08.440046 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7731fb8-fd8a-40ed-9e01-6bf61cff9db0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:08.440365 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7731fb8-fd8a-40ed-9e01-6bf61cff9db0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:08.442269 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7731fb8-fd8a-40ed-9e01-6bf61cff9db0 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:08.456654 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 268/541] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Oct 22 23:56:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:08.457797 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7731fb8-fd8a-40ed-9e01-6bf61cff9db0 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:08.484455 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8dec5-209e-4f15-92b5-78ae8c0aeae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:08.485137 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8dec5-209e-4f15-92b5-78ae8c0aeae3 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:08.485523 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8dec5-209e-4f15-92b5-78ae8c0aeae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:08.485803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8dec5-209e-4f15-92b5-78ae8c0aeae3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:08.496389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b8dec5-209e-4f15-92b5-78ae8c0aeae3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:08.497892 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b8dec5-209e-4f15-92b5-78ae8c0aeae3 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:08.516013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-86b8dec5-209e-4f15-92b5-78ae8c0aeae3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:08.516013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:08.516013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:08.516013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:08.516013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:08.516013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:08.516013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:08.516013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:08.516013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:08.516013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:08.516013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:08.516013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:08.516013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:08.516013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:08.516013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:08.516013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:08.517857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:08.517857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:08.517857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:08.517857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:08.517857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:08.517857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:08.517857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:08.517857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:08.517857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:08.517857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:08.517857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:08.517857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:08.517857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:08.517857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:08.517857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:08.517857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:08.517857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:08.518737 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:08.518737 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:08.518737 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:08.522285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 274/542] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:08.523168 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8dec5-209e-4f15-92b5-78ae8c0aeae3 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:08.555886 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:08.580325 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7731fb8-fd8a-40ed-9e01-6bf61cff9db0 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:08.618864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33cf03-fcaf-4550-8238-5e59c2f86a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:08.619799 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33cf03-fcaf-4550-8238-5e59c2f86a14 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:08.620083 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33cf03-fcaf-4550-8238-5e59c2f86a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:08.620355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33cf03-fcaf-4550-8238-5e59c2f86a14 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:08.621057 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee33cf03-fcaf-4550-8238-5e59c2f86a14 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:08.622526 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee33cf03-fcaf-4550-8238-5e59c2f86a14 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:08.640171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 269/543] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Oct 22 23:56:08 2022] POST /identity/v3/services => generated 251 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:56:08.641451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504049d6-ba39-4c29-b394-b97a1cbfc7a5 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:08.642311 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504049d6-ba39-4c29-b394-b97a1cbfc7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:08.642592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504049d6-ba39-4c29-b394-b97a1cbfc7a5 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:08.644076 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:08.644197 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 270/544] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:08.644929 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504049d6-ba39-4c29-b394-b97a1cbfc7a5 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:08.654851 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8dec5-209e-4f15-92b5-78ae8c0aeae3 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:08.723185 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58be7f3-55c4-4437-aaff-8143d9076c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:08.725955 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58be7f3-55c4-4437-aaff-8143d9076c74 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:08.726467 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58be7f3-55c4-4437-aaff-8143d9076c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:08.726742 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58be7f3-55c4-4437-aaff-8143d9076c74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:08.744747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58be7f3-55c4-4437-aaff-8143d9076c74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:08.747295 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58be7f3-55c4-4437-aaff-8143d9076c74 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:08.765765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-f58be7f3-55c4-4437-aaff-8143d9076c74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:08.765765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:08.765765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:08.765765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:08.765765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:08.765765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:08.765765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:08.765765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:08.765765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:08.765765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:08.765765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:08.765765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:08.765765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:08.765765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:08.765765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:08.765765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:08.767497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:08.767497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:08.767497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:08.767497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:08.767497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:08.767497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:08.767497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:08.767497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:08.767497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:08.767497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:08.767497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:08.767497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:08.767497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:08.767497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:08.767497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:08.767497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:08.767497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:08.768400 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:08.768400 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:08.768400 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:08.768400 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 275/545] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:08.774286 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58be7f3-55c4-4437-aaff-8143d9076c74 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:08.802607 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504049d6-ba39-4c29-b394-b97a1cbfc7a5 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:08.843043 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d779f-bd69-489d-a903-5f7e6d95d484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:08.844070 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d779f-bd69-489d-a903-5f7e6d95d484 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:08.845111 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d779f-bd69-489d-a903-5f7e6d95d484 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:08.845675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d779f-bd69-489d-a903-5f7e6d95d484 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:08.846056 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d779f-bd69-489d-a903-5f7e6d95d484 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:08.846364 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d779f-bd69-489d-a903-5f7e6d95d484 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:08.847940 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d779f-bd69-489d-a903-5f7e6d95d484 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:08.855875 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:08.864647 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 271/546] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:08.866273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfbacc-99e5-4d32-98b0-3b85b7f8bdea None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:08.866758 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfbacc-99e5-4d32-98b0-3b85b7f8bdea None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:08.867433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfbacc-99e5-4d32-98b0-3b85b7f8bdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:08.887393 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-8ddfbacc-99e5-4d32-98b0-3b85b7f8bdea None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:08.888998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8ddfbacc-99e5-4d32-98b0-3b85b7f8bdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:08.898467 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58be7f3-55c4-4437-aaff-8143d9076c74 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:08.931941 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3c812-6d4b-47a0-8a6a-25c95ad78a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:08.932595 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3c812-6d4b-47a0-8a6a-25c95ad78a99 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:08.932884 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3c812-6d4b-47a0-8a6a-25c95ad78a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:08.933161 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3c812-6d4b-47a0-8a6a-25c95ad78a99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:08.943474 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f3c812-6d4b-47a0-8a6a-25c95ad78a99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:56:08.948760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-27f3c812-6d4b-47a0-8a6a-25c95ad78a99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:56:08.948760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:08.948760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:08.948760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:08.948760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:08.948760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:08.948760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:08.948760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:08.948760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:08.948760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:08.948760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:08.948760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:08.948760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:56:08.948760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:56:08.948760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:56:08.948760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:56:08.949902 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:56:08.949902 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:56:08.949902 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:56:08.949902 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:56:08.949902 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:08.949902 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:08.949902 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:08.949902 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:08.949902 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:08.949902 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:08.949902 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:08.949902 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:08.949902 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:08.949902 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:08.949902 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:08.949902 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:08.950946 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:08.950946 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:08.950946 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:08.950946 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:08.950946 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:56:08.950946 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:56:08.950946 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:56:08.950946 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:56:08.950946 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:56:08.950946 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:56:08.950946 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:56:08.950946 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:08.955437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 276/547] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Oct 22 23:56:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:08.955914 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3c812-6d4b-47a0-8a6a-25c95ad78a99 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:08.959423 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:08.971141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 272/548] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:08 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 22 23:56:08.976805 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfbacc-99e5-4d32-98b0-3b85b7f8bdea None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:09.073780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3c812-6d4b-47a0-8a6a-25c95ad78a99 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:09.091530 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfbacc-99e5-4d32-98b0-3b85b7f8bdea None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:09.112953 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad38bf-920f-4b34-827e-2b88b53dc27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:09.113564 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad38bf-920f-4b34-827e-2b88b53dc27f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:09.113852 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad38bf-920f-4b34-827e-2b88b53dc27f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:09.114129 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad38bf-920f-4b34-827e-2b88b53dc27f None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:09.114773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad38bf-920f-4b34-827e-2b88b53dc27f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:09.115282 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad38bf-920f-4b34-827e-2b88b53dc27f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:09.117306 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad38bf-920f-4b34-827e-2b88b53dc27f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:09.130317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:09.134130 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 277/549] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:09.135183 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad38bf-920f-4b34-827e-2b88b53dc27f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:09.140454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b696588-b92f-4a96-ba26-c3c3ffc53bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:09.140859 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b696588-b92f-4a96-ba26-c3c3ffc53bcf None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:09.141236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b696588-b92f-4a96-ba26-c3c3ffc53bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:09.142256 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b696588-b92f-4a96-ba26-c3c3ffc53bcf None admin] PATH_INFO: `/v3/users/nova` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:09.150866 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b696588-b92f-4a96-ba26-c3c3ffc53bcf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:09.152195 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b696588-b92f-4a96-ba26-c3c3ffc53bcf None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:09.165103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-6b696588-b92f-4a96-ba26-c3c3ffc53bcf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 22 23:56:09.165103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:09.165103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:09.165103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:09.165103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:09.165103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:09.165103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:09.165103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:09.165103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:09.165103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:09.165103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:09.165103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:09.165103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:56:09.165103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:56:09.165103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:56:09.165103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:56:09.166148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:09.166148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:09.166148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:56:09.166148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:09.166148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:56:09.166148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:09.166148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:09.166148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:09.166148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:09.166148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:09.166148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:09.166148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:09.166148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:09.166148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:09.166148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:09.166148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:09.166148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:09.166148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:09.167109 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:09.167109 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:09.167109 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:56:09.167109 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:56:09.167109 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:56:09.167109 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:56:09.167109 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:56:09.167109 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:56:09.167109 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 22 23:56:09.167109 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:09.167109 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 273/550] 10.222.0.13 () {56 vars in 1093 bytes} [Sat Oct 22 23:56:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:09.167738 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b696588-b92f-4a96-ba26-c3c3ffc53bcf None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:09.211421 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:09.247601 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad38bf-920f-4b34-827e-2b88b53dc27f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:09.272250 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b696588-b92f-4a96-ba26-c3c3ffc53bcf None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:09.285663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0807f-78fb-4421-8615-3bd28dabf41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:09.286273 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0807f-78fb-4421-8615-3bd28dabf41d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:09.286555 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0807f-78fb-4421-8615-3bd28dabf41d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:09.286828 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0807f-78fb-4421-8615-3bd28dabf41d None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:09.287485 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e0807f-78fb-4421-8615-3bd28dabf41d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:09.287784 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e0807f-78fb-4421-8615-3bd28dabf41d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:09.289174 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e0807f-78fb-4421-8615-3bd28dabf41d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:09.297568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 278/551] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:56:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:09.303024 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0807f-78fb-4421-8615-3bd28dabf41d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:09.308176 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68495a42-6e49-403d-ba1d-02996ec49166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:09.308835 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68495a42-6e49-403d-ba1d-02996ec49166 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:09.309116 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68495a42-6e49-403d-ba1d-02996ec49166 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:09.309387 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68495a42-6e49-403d-ba1d-02996ec49166 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:09.318744 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68495a42-6e49-403d-ba1d-02996ec49166 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:09.320172 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68495a42-6e49-403d-ba1d-02996ec49166 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:09.332637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-68495a42-6e49-403d-ba1d-02996ec49166 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 22 23:56:09.332637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:09.332637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:09.332637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:09.332637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:09.332637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:09.332637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:09.332637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:09.332637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:09.332637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:09.332637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:09.332637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:09.332637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:56:09.332637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:56:09.332637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:56:09.332637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:56:09.333630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:09.333630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:09.333630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:56:09.333630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:09.333630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:56:09.333630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:09.333630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:09.333630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:09.333630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:09.333630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:09.333630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:09.333630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:09.333630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:09.333630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:09.333630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:09.333630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:09.333630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:09.333630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:09.334592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:09.334592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:09.334592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:56:09.334592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:56:09.334592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:56:09.334592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:56:09.334592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:56:09.334592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:56:09.334592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 22 23:56:09.334592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:09.334592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 274/552] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Oct 22 23:56:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:09.334592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68495a42-6e49-403d-ba1d-02996ec49166 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:09.387958 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:09.415748 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0807f-78fb-4421-8615-3bd28dabf41d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:09.443608 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:09.443948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68495a42-6e49-403d-ba1d-02996ec49166 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:09.453315 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee29cf-b4e3-4a8b-824b-d01782a4c6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:09.453919 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee29cf-b4e3-4a8b-824b-d01782a4c6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:09.454772 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee29cf-b4e3-4a8b-824b-d01782a4c6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:09.455623 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee29cf-b4e3-4a8b-824b-d01782a4c6b0 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:09.456354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee29cf-b4e3-4a8b-824b-d01782a4c6b0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:09.456689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee29cf-b4e3-4a8b-824b-d01782a4c6b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:09.458088 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee29cf-b4e3-4a8b-824b-d01782a4c6b0 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:09.478679 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 279/553] 10.222.0.13 () {56 vars in 1133 bytes} [Sat Oct 22 23:56:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:09.479336 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee29cf-b4e3-4a8b-824b-d01782a4c6b0 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:09.486036 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e342b20-f383-4bd2-80a9-d660884f560d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:09.486813 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e342b20-f383-4bd2-80a9-d660884f560d None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:09.487128 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e342b20-f383-4bd2-80a9-d660884f560d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:09.487406 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e342b20-f383-4bd2-80a9-d660884f560d None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:09.488061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e342b20-f383-4bd2-80a9-d660884f560d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:09.489467 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e342b20-f383-4bd2-80a9-d660884f560d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:09.505144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 275/554] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Oct 22 23:56:09 2022] POST /identity/v3/services => generated 240 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:56:09.514322 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e342b20-f383-4bd2-80a9-d660884f560d None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:09.606118 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee29cf-b4e3-4a8b-824b-d01782a4c6b0 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:09.661547 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e342b20-f383-4bd2-80a9-d660884f560d None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:09.665139 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:09.665509 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59e772-b1e6-4f47-b20e-288167dd28fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:09.666295 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59e772-b1e6-4f47-b20e-288167dd28fa None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:09.666752 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59e772-b1e6-4f47-b20e-288167dd28fa None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:09.666930 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59e772-b1e6-4f47-b20e-288167dd28fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:09.686628 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e59e772-b1e6-4f47-b20e-288167dd28fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:09.688611 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e59e772-b1e6-4f47-b20e-288167dd28fa None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:09.706069 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506682f6-3e94-4e33-bfb2-5d12b863d477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:09.706687 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506682f6-3e94-4e33-bfb2-5d12b863d477 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:09.706972 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506682f6-3e94-4e33-bfb2-5d12b863d477 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:09.707248 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506682f6-3e94-4e33-bfb2-5d12b863d477 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:09.707984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506682f6-3e94-4e33-bfb2-5d12b863d477 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:09.708295 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506682f6-3e94-4e33-bfb2-5d12b863d477 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:09.709757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506682f6-3e94-4e33-bfb2-5d12b863d477 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:09.710160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-4e59e772-b1e6-4f47-b20e-288167dd28fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:09.710160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:09.710160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:09.710160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:09.710160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:09.710160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:09.710160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:09.710160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:09.710160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:09.710160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:09.710160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:09.710160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:09.710160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:09.710160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:09.710160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:09.710160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:09.711066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:09.711066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:09.711066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:09.711066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:09.711066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:09.711066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:09.711066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:09.711066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:09.711066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:09.711066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:09.711066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:09.711066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:09.711066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:09.711066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:09.711066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:09.711066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:09.711066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:09.711998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:09.711998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:09.711998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:09.716527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 280/555] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:09.716527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59e772-b1e6-4f47-b20e-288167dd28fa None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:09.732726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 276/556] 10.222.0.13 () {56 vars in 1149 bytes} [Sat Oct 22 23:56:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:09.733559 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506682f6-3e94-4e33-bfb2-5d12b863d477 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:09.753959 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:09.833260 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59e772-b1e6-4f47-b20e-288167dd28fa None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:09.871639 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506682f6-3e94-4e33-bfb2-5d12b863d477 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:09.874010 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f4e8d-728a-4615-aa8f-98ad7b70394e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:09.875277 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f4e8d-728a-4615-aa8f-98ad7b70394e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:09.875277 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f4e8d-728a-4615-aa8f-98ad7b70394e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:09.875277 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f4e8d-728a-4615-aa8f-98ad7b70394e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:09.918133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd1494-d3b3-4907-9082-d7b5c7f29bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:09.919092 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd1494-d3b3-4907-9082-d7b5c7f29bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:09.920009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd1494-d3b3-4907-9082-d7b5c7f29bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:09.920540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd1494-d3b3-4907-9082-d7b5c7f29bc8 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:09.921385 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd1494-d3b3-4907-9082-d7b5c7f29bc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:09.922238 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd1494-d3b3-4907-9082-d7b5c7f29bc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:09.924617 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd1494-d3b3-4907-9082-d7b5c7f29bc8 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:09.939869 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 277/557] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:09.941117 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd1494-d3b3-4907-9082-d7b5c7f29bc8 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:09.961331 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f4e8d-728a-4615-aa8f-98ad7b70394e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:09.962784 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f4e8d-728a-4615-aa8f-98ad7b70394e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:09.982535 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:09.996760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:10.072829 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 281/558] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:56:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 22 23:56:10.078066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f4e8d-728a-4615-aa8f-98ad7b70394e None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:10.085005 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd1494-d3b3-4907-9082-d7b5c7f29bc8 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:10.138136 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10915be2-1515-4f01-bc31-aee21f5c1a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:10.138992 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10915be2-1515-4f01-bc31-aee21f5c1a5a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:10.139377 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10915be2-1515-4f01-bc31-aee21f5c1a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:10.139992 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10915be2-1515-4f01-bc31-aee21f5c1a5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:10.154713 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10915be2-1515-4f01-bc31-aee21f5c1a5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:10.156193 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10915be2-1515-4f01-bc31-aee21f5c1a5a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:10.171588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-10915be2-1515-4f01-bc31-aee21f5c1a5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:10.171588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:10.171588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:10.171588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:10.171588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:10.171588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:10.171588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:10.171588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:10.171588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:10.171588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:10.171588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:10.171588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:10.171588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:10.171588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:10.171588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:10.171588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:10.172631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:10.172631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:10.172631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:10.172631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:10.172631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:10.172631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:10.172631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:10.172631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:10.172631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:10.172631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:10.172631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:10.172631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:10.172631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:10.172631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:10.172631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:10.172631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:10.172631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:10.174160 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:10.174160 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:10.174160 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:10.174160 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 278/559] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:10.174160 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10915be2-1515-4f01-bc31-aee21f5c1a5a None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:10.218562 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f4e8d-728a-4615-aa8f-98ad7b70394e None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:10.252078 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:10.262896 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:10.272274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7a104-2448-4697-bd70-bfc6cb98e2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:10.273928 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7a104-2448-4697-bd70-bfc6cb98e2d5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:10.274580 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7a104-2448-4697-bd70-bfc6cb98e2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:10.275149 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7a104-2448-4697-bd70-bfc6cb98e2d5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:10.282974 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10915be2-1515-4f01-bc31-aee21f5c1a5a None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:10.286295 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7a104-2448-4697-bd70-bfc6cb98e2d5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:10.287964 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7a104-2448-4697-bd70-bfc6cb98e2d5 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:10.304513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-5ec7a104-2448-4697-bd70-bfc6cb98e2d5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 22 23:56:10.304513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:10.304513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:10.304513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:10.304513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:10.304513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:10.304513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:10.304513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:10.304513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:10.304513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:10.304513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:10.304513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:10.304513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:56:10.304513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:56:10.304513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:56:10.304513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:56:10.305525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:10.305525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:10.305525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:56:10.305525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:10.305525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:56:10.305525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:10.305525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:10.305525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:10.305525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:10.305525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:10.305525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:10.305525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:10.305525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:10.305525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:10.305525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:10.305525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:10.305525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:10.305525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:10.306532 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:10.306532 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:10.306532 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:56:10.306532 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:56:10.306532 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:56:10.306532 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:56:10.306532 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:56:10.306532 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:56:10.306532 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 22 23:56:10.306532 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:10.309651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 282/560] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Oct 22 23:56:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:10.310597 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7a104-2448-4697-bd70-bfc6cb98e2d5 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:10.328167 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94033b6-e12c-4233-827a-a37ce6a6c76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:10.328801 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94033b6-e12c-4233-827a-a37ce6a6c76f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:10.329847 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94033b6-e12c-4233-827a-a37ce6a6c76f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:10.330156 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94033b6-e12c-4233-827a-a37ce6a6c76f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:10.342030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94033b6-e12c-4233-827a-a37ce6a6c76f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:10.343597 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94033b6-e12c-4233-827a-a37ce6a6c76f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:10.358996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-a94033b6-e12c-4233-827a-a37ce6a6c76f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:10.358996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:10.358996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:10.358996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:10.358996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:10.358996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:10.358996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:10.358996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:10.358996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:10.358996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:10.358996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:10.358996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:10.358996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:10.358996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:10.358996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:10.358996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:10.360170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:10.360170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:10.360170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:10.360170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:10.360170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:10.360170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:10.360170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:10.360170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:10.360170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:10.360170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:10.360170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:10.360170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:10.360170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:10.360170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:10.360170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:10.360170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:10.360170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:10.361848 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:10.361848 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:10.361848 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:10.367674 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 279/561] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:10.368907 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94033b6-e12c-4233-827a-a37ce6a6c76f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:10.440276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7a104-2448-4697-bd70-bfc6cb98e2d5 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:10.477664 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e3be3-0d56-4a00-8416-287eec34fbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:10.478437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e3be3-0d56-4a00-8416-287eec34fbea None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:10.478874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e3be3-0d56-4a00-8416-287eec34fbea None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:10.479249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e3be3-0d56-4a00-8416-287eec34fbea None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:10.479996 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e3be3-0d56-4a00-8416-287eec34fbea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:10.480504 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e3be3-0d56-4a00-8416-287eec34fbea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:10.481898 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e3be3-0d56-4a00-8416-287eec34fbea None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:10.497304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 283/562] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:10.498368 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e3be3-0d56-4a00-8416-287eec34fbea None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:10.503769 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94033b6-e12c-4233-827a-a37ce6a6c76f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:10.541350 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:10.554102 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570d376-fb99-40d6-b554-b5650b089103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:10.555199 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570d376-fb99-40d6-b554-b5650b089103 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:10.555632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570d376-fb99-40d6-b554-b5650b089103 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:10.556439 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570d376-fb99-40d6-b554-b5650b089103 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:10.557348 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570d376-fb99-40d6-b554-b5650b089103 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:10.557821 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570d376-fb99-40d6-b554-b5650b089103 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:10.559848 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:10.560949 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570d376-fb99-40d6-b554-b5650b089103 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:10.584789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 280/563] 10.222.0.13 () {56 vars in 1135 bytes} [Sat Oct 22 23:56:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:10.585891 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570d376-fb99-40d6-b554-b5650b089103 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:10.612631 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e3be3-0d56-4a00-8416-287eec34fbea None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:10.649651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998ae84-3184-43a2-a56a-54951e1ea205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:10.650786 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998ae84-3184-43a2-a56a-54951e1ea205 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:10.651002 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998ae84-3184-43a2-a56a-54951e1ea205 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:10.651380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998ae84-3184-43a2-a56a-54951e1ea205 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:10.652121 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998ae84-3184-43a2-a56a-54951e1ea205 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:10.652546 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998ae84-3184-43a2-a56a-54951e1ea205 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:10.654050 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998ae84-3184-43a2-a56a-54951e1ea205 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:10.671650 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 284/564] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:10.673128 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998ae84-3184-43a2-a56a-54951e1ea205 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:10.693543 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570d376-fb99-40d6-b554-b5650b089103 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:10.729451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0fb73b-2d47-4037-8a74-d0d14aeb703c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:10.730324 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fb73b-2d47-4037-8a74-d0d14aeb703c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:10.730752 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fb73b-2d47-4037-8a74-d0d14aeb703c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:10.731174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0fb73b-2d47-4037-8a74-d0d14aeb703c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:10.738397 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0fb73b-2d47-4037-8a74-d0d14aeb703c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:10.740057 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0fb73b-2d47-4037-8a74-d0d14aeb703c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:10.752699 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-bc0fb73b-2d47-4037-8a74-d0d14aeb703c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:10.752699 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:10.752699 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:10.752699 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:10.752699 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:10.752699 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:10.752699 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:10.752699 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:10.752699 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:10.752699 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:10.752699 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:10.752699 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:10.752699 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:56:10.752699 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:56:10.752699 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:56:10.753623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:56:10.753623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:10.753623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:10.753623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:10.753623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:10.753623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:10.753623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:10.753623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:10.753623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:10.753623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:10.753623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:10.753623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:10.753623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:10.753623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:10.753623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:10.753623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:10.753623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:10.754537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:56:10.754537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:56:10.754537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:56:10.754537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:56:10.754537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:56:10.754537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:56:10.754537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:10.754537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:10.756930 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 281/565] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:56:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:10.757906 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0fb73b-2d47-4037-8a74-d0d14aeb703c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:10.784338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998ae84-3184-43a2-a56a-54951e1ea205 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:10.799869 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:10.809128 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:10.829949 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e235c-0312-43f4-a0fc-569dd5fb5dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:10.830636 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e235c-0312-43f4-a0fc-569dd5fb5dbd None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:10.830879 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e235c-0312-43f4-a0fc-569dd5fb5dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:10.831179 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e235c-0312-43f4-a0fc-569dd5fb5dbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:10.842901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e235c-0312-43f4-a0fc-569dd5fb5dbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:10.844563 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e235c-0312-43f4-a0fc-569dd5fb5dbd None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:10.860787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-dc3e235c-0312-43f4-a0fc-569dd5fb5dbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:10.860787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:10.860787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:10.860787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:10.860787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:10.860787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:10.860787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:10.860787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:10.860787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:10.860787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:10.860787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:10.860787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:10.860787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:10.860787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:10.860787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:10.860787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:10.861846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:10.861846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:10.861846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:10.861846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:10.861846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:10.861846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:10.861846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:10.861846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:10.861846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:10.861846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:10.861846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:10.861846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:10.861846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:10.861846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:10.861846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:10.861846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:10.861846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:10.862877 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:10.862877 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:10.862877 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:10.870846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 285/566] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:10.871819 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e235c-0312-43f4-a0fc-569dd5fb5dbd None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:10.873779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0fb73b-2d47-4037-8a74-d0d14aeb703c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:10.930167 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b524cf9-b36e-45ba-9a0d-9b03a2f2ebac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:10.930801 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b524cf9-b36e-45ba-9a0d-9b03a2f2ebac None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:10.931084 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b524cf9-b36e-45ba-9a0d-9b03a2f2ebac None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:10.931359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b524cf9-b36e-45ba-9a0d-9b03a2f2ebac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:10.938209 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b524cf9-b36e-45ba-9a0d-9b03a2f2ebac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:10.939704 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b524cf9-b36e-45ba-9a0d-9b03a2f2ebac None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:10.951177 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-9b524cf9-b36e-45ba-9a0d-9b03a2f2ebac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:10.951177 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:10.951177 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:10.951177 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:10.951177 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:10.951177 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:10.951177 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:10.951177 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:10.951177 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:10.951177 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:10.951177 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:10.951177 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:10.951177 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:56:10.951177 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:56:10.951177 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:56:10.952130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:56:10.952130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:10.952130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:10.952130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:10.952130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:10.952130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:10.952130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:10.952130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:10.952130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:10.952130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:10.952130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:10.952130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:10.952130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:10.952130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:10.952130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:10.952130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:10.952130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:10.953066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:56:10.953066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:56:10.953066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:56:10.953066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:56:10.953066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:56:10.953066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:56:10.953066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:10.953066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:10.953066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 282/567] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:56:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:10.953066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b524cf9-b36e-45ba-9a0d-9b03a2f2ebac None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:10.985698 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e235c-0312-43f4-a0fc-569dd5fb5dbd None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:11.021113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2dfd4-fad2-4bea-b449-6839bf3c308c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:11.021791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2dfd4-fad2-4bea-b449-6839bf3c308c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:11.021995 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2dfd4-fad2-4bea-b449-6839bf3c308c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:11.022269 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2dfd4-fad2-4bea-b449-6839bf3c308c None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:11.022960 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee2dfd4-fad2-4bea-b449-6839bf3c308c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:11.023247 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee2dfd4-fad2-4bea-b449-6839bf3c308c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:11.024692 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee2dfd4-fad2-4bea-b449-6839bf3c308c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:11.037300 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 286/568] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Oct 22 23:56:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:11.038281 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fbb6c-47a7-461d-baba-ce9ad93812b3 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:11.038570 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fbb6c-47a7-461d-baba-ce9ad93812b3 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:11.038846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fbb6c-47a7-461d-baba-ce9ad93812b3 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:11.044575 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 287/569] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:11.044575 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2fbb6c-47a7-461d-baba-ce9ad93812b3 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:11.063758 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b524cf9-b36e-45ba-9a0d-9b03a2f2ebac None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:11.066174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:11.073020 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:11.099230 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc98a4e-82dc-4e42-bc26-4efa2f5af43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:11.099836 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc98a4e-82dc-4e42-bc26-4efa2f5af43f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:11.100121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc98a4e-82dc-4e42-bc26-4efa2f5af43f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:11.100395 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc98a4e-82dc-4e42-bc26-4efa2f5af43f None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:11.103690 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc98a4e-82dc-4e42-bc26-4efa2f5af43f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:11.103690 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc98a4e-82dc-4e42-bc26-4efa2f5af43f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:11.103690 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc98a4e-82dc-4e42-bc26-4efa2f5af43f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:11.121731 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 283/570] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:11.121731 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc98a4e-82dc-4e42-bc26-4efa2f5af43f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:11.165319 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2fbb6c-47a7-461d-baba-ce9ad93812b3 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:11.207894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c1b4a-af4c-4c85-82a3-b7fab35c7896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:11.208528 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c1b4a-af4c-4c85-82a3-b7fab35c7896 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:11.208826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c1b4a-af4c-4c85-82a3-b7fab35c7896 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:11.209046 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c1b4a-af4c-4c85-82a3-b7fab35c7896 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:11.209784 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c1b4a-af4c-4c85-82a3-b7fab35c7896 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:11.210022 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c1b4a-af4c-4c85-82a3-b7fab35c7896 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:11.211457 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c1b4a-af4c-4c85-82a3-b7fab35c7896 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:11.222709 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc98a4e-82dc-4e42-bc26-4efa2f5af43f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:11.228037 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 288/571] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Oct 22 23:56:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:11.228037 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6df969-2633-4151-93bf-7bb43464ab33 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:11.228037 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6df969-2633-4151-93bf-7bb43464ab33 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:11.228037 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6df969-2633-4151-93bf-7bb43464ab33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:11.243854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1f6df969-2633-4151-93bf-7bb43464ab33 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:11.245054 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1f6df969-2633-4151-93bf-7bb43464ab33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:11.264053 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0b226-aee8-4bf6-9351-df499b2e34fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:11.264866 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0b226-aee8-4bf6-9351-df499b2e34fd None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:11.265154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0b226-aee8-4bf6-9351-df499b2e34fd None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:11.265428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0b226-aee8-4bf6-9351-df499b2e34fd None admin] PATH_INFO: `/v3/projects/5c9246c059754504afee4711d113a3fe/users/7c0356072f92451f98b03a2225e22f86/roles/73721a054c3842bba868ab52c1a3f8e0` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:11.281372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0b226-aee8-4bf6-9351-df499b2e34fd None admin] RBAC: Authorizing `identity:create_grant(project_id=5c9246c059754504afee4711d113a3fe, user_id=7c0356072f92451f98b03a2225e22f86, role_id=73721a054c3842bba868ab52c1a3f8e0)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:11.282905 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0b226-aee8-4bf6-9351-df499b2e34fd None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:11.304532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 284/572] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:56:11 2022] PUT /identity/v3/projects/5c9246c059754504afee4711d113a3fe/users/7c0356072f92451f98b03a2225e22f86/roles/73721a054c3842bba868ab52c1a3f8e0 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:56:11.304532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0b226-aee8-4bf6-9351-df499b2e34fd None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:11.323282 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:11.323494 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 289/573] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:11.324065 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6df969-2633-4151-93bf-7bb43464ab33 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:11.341139 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:11.478361 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0b226-aee8-4bf6-9351-df499b2e34fd None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:11.520959 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a13d1b-17bc-4774-aa69-01c0650048dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:11.523075 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a13d1b-17bc-4774-aa69-01c0650048dd None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:11.523684 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a13d1b-17bc-4774-aa69-01c0650048dd None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:11.524192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a13d1b-17bc-4774-aa69-01c0650048dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:11.533478 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a13d1b-17bc-4774-aa69-01c0650048dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:11.535544 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a13d1b-17bc-4774-aa69-01c0650048dd None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:11.542672 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6df969-2633-4151-93bf-7bb43464ab33 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:11.552327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-17a13d1b-17bc-4774-aa69-01c0650048dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:11.552327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:11.552327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:11.552327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:11.552327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:11.552327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:11.552327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:11.552327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:11.552327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:11.552327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:11.552327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:11.552327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:11.552327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:56:11.552327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:56:11.552327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:56:11.553315 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:56:11.553315 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:11.553315 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:11.553315 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:11.553315 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:11.553315 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:11.553315 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:11.553315 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:11.553315 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:11.553315 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:11.553315 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:11.553315 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:11.553315 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:11.553315 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:11.553315 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:11.553315 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:11.553315 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:11.554218 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:56:11.554218 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:56:11.554218 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:56:11.554218 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:56:11.554218 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:56:11.554218 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:56:11.554218 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:11.554218 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:11.557228 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 285/574] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:56:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:11.563430 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a13d1b-17bc-4774-aa69-01c0650048dd None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:11.597384 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973379fe-e24b-4ffe-8a5a-50a6e99544e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:11.598076 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973379fe-e24b-4ffe-8a5a-50a6e99544e4 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:11.598423 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973379fe-e24b-4ffe-8a5a-50a6e99544e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:11.598763 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973379fe-e24b-4ffe-8a5a-50a6e99544e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:11.599564 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973379fe-e24b-4ffe-8a5a-50a6e99544e4 None admin] RBAC: Adding query filter params (role.id=839a55263e2c4cbaa2f945b75cf298d6, scope.project.id=5c9246c059754504afee4711d113a3fe, user.id=c4a3579d56294b9d8ad36d4d174c8526) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:11.599938 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973379fe-e24b-4ffe-8a5a-50a6e99544e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:11.601458 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973379fe-e24b-4ffe-8a5a-50a6e99544e4 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:11.607810 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 290/575] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:56:11 2022] GET /identity/v3/role_assignments?user.id=c4a3579d56294b9d8ad36d4d174c8526&scope.project.id=5c9246c059754504afee4711d113a3fe&role.id=839a55263e2c4cbaa2f945b75cf298d6 => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:11.632844 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973379fe-e24b-4ffe-8a5a-50a6e99544e4 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:11.652004 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:11.683596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a13d1b-17bc-4774-aa69-01c0650048dd None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:11.739551 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5c00c-27c4-4d4a-adc8-1a1c9ccb3661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:11.740594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5c00c-27c4-4d4a-adc8-1a1c9ccb3661 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:11.741099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5c00c-27c4-4d4a-adc8-1a1c9ccb3661 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:11.741614 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b5c00c-27c4-4d4a-adc8-1a1c9ccb3661 None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:11.742575 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5c00c-27c4-4d4a-adc8-1a1c9ccb3661 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:11.743108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5c00c-27c4-4d4a-adc8-1a1c9ccb3661 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:11.744860 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b5c00c-27c4-4d4a-adc8-1a1c9ccb3661 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:11.753695 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:11.770994 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 286/576] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Oct 22 23:56:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:11.776820 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5c00c-27c4-4d4a-adc8-1a1c9ccb3661 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:11.802474 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973379fe-e24b-4ffe-8a5a-50a6e99544e4 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:11.866805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24444b3-1f5d-4be4-b3ec-c6ec7538e89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:11.867509 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24444b3-1f5d-4be4-b3ec-c6ec7538e89a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:11.867799 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24444b3-1f5d-4be4-b3ec-c6ec7538e89a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:11.868079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24444b3-1f5d-4be4-b3ec-c6ec7538e89a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:11.869522 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24444b3-1f5d-4be4-b3ec-c6ec7538e89a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:11.871385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24444b3-1f5d-4be4-b3ec-c6ec7538e89a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:11.883083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-d24444b3-1f5d-4be4-b3ec-c6ec7538e89a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 22 23:56:11.883083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:11.883083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:56:11.883083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:56:11.883083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:56:11.883083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:56:11.883083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:56:11.883083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:11.883083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 22 23:56:11.883083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:11.883083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:11.883083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:11.883083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:11.883083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:11.883083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:11.883083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:11.883083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:11.884141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:11.884141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:11.884141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:11.884141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:11.884141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:11.884141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:11.884141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:56:11.884141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:56:11.884141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:56:11.884141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:56:11.884141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:11.884141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:11.884141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:11.884141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:11.884141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:11.884141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:11.885059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:11.885059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:11.885059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:11.885059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:11.885059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:11.885059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:11.885059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:11.885059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:11.885059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:11.885059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:11.885059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:56:11.885059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:11.885059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 22 23:56:11.885059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:11.885059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 291/577] 10.222.0.13 () {56 vars in 1129 bytes} [Sat Oct 22 23:56:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:56:11.885925 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634a459-aa4f-44de-9ae7-d42e1fa1614b None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:11.885925 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634a459-aa4f-44de-9ae7-d42e1fa1614b None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:11.886672 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634a459-aa4f-44de-9ae7-d42e1fa1614b None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:11.887521 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 292/578] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:11.893636 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e9a5c-4e7e-4694-ab31-422e178bd3e5 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:11.893919 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e9a5c-4e7e-4694-ab31-422e178bd3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:11.894201 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e9a5c-4e7e-4694-ab31-422e178bd3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:11.922198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-a78e9a5c-4e7e-4694-ab31-422e178bd3e5 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:11.924579 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a78e9a5c-4e7e-4694-ab31-422e178bd3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:11.936647 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b5c00c-27c4-4d4a-adc8-1a1c9ccb3661 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:11.986479 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4134c1d-d915-4f88-b3f9-eedecbcf9f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:11.987718 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4134c1d-d915-4f88-b3f9-eedecbcf9f1c None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:11.988481 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4134c1d-d915-4f88-b3f9-eedecbcf9f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:11.989011 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4134c1d-d915-4f88-b3f9-eedecbcf9f1c None admin] PATH_INFO: `/v3/projects/5c9246c059754504afee4711d113a3fe/users/bfd8c7516b7c435d88298672b2596d6d/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:11.991669 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4134c1d-d915-4f88-b3f9-eedecbcf9f1c None admin] RBAC: Authorizing `identity:create_grant(project_id=5c9246c059754504afee4711d113a3fe, user_id=bfd8c7516b7c435d88298672b2596d6d, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:11.993935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4134c1d-d915-4f88-b3f9-eedecbcf9f1c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:11.997606 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:12.009583 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 293/579] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 22 23:56:12.013922 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 287/580] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:56:11 2022] PUT /identity/v3/projects/5c9246c059754504afee4711d113a3fe/users/bfd8c7516b7c435d88298672b2596d6d/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 22 23:56:12.017823 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e9a5c-4e7e-4694-ab31-422e178bd3e5 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:12.103762 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:12.133979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e9a5c-4e7e-4694-ab31-422e178bd3e5 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:12.170944 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769eaa73-a974-4001-90d0-696f2690fb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:12.171567 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eaa73-a974-4001-90d0-696f2690fb7f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:12.171853 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eaa73-a974-4001-90d0-696f2690fb7f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:12.172134 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eaa73-a974-4001-90d0-696f2690fb7f None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:12.172878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769eaa73-a974-4001-90d0-696f2690fb7f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:12.173227 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769eaa73-a974-4001-90d0-696f2690fb7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:12.174813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769eaa73-a974-4001-90d0-696f2690fb7f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:12.180467 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 294/581] 10.222.0.13 () {56 vars in 1124 bytes} [Sat Oct 22 23:56:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:12.186115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4134c1d-d915-4f88-b3f9-eedecbcf9f1c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:12.326262 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4134c1d-d915-4f88-b3f9-eedecbcf9f1c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:12.364929 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a93968-e918-4f76-a64f-55e0703ca709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:12.365545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a93968-e918-4f76-a64f-55e0703ca709 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:12.365831 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a93968-e918-4f76-a64f-55e0703ca709 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:12.366111 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a93968-e918-4f76-a64f-55e0703ca709 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:12.366742 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a93968-e918-4f76-a64f-55e0703ca709 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:12.367047 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a93968-e918-4f76-a64f-55e0703ca709 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:12.368522 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a93968-e918-4f76-a64f-55e0703ca709 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:12.369241 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769eaa73-a974-4001-90d0-696f2690fb7f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:12.374233 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 288/582] 10.222.0.13 () {56 vars in 1124 bytes} [Sat Oct 22 23:56:12 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:12.384115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a93968-e918-4f76-a64f-55e0703ca709 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:12.472274 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:12.509454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769eaa73-a974-4001-90d0-696f2690fb7f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:12.562984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a93968-e918-4f76-a64f-55e0703ca709 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:12.567664 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17897d44-084a-472f-9284-b8a050c9c248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:12.569084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17897d44-084a-472f-9284-b8a050c9c248 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:12.570165 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17897d44-084a-472f-9284-b8a050c9c248 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:12.570820 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17897d44-084a-472f-9284-b8a050c9c248 None admin] PATH_INFO: `/v3/services/network` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:12.571599 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17897d44-084a-472f-9284-b8a050c9c248 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:12.577365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17897d44-084a-472f-9284-b8a050c9c248 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:12.587673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-17897d44-084a-472f-9284-b8a050c9c248 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 22 23:56:12.587673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:12.587673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:56:12.587673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:56:12.587673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:56:12.587673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:56:12.587673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:56:12.587673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:12.587673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 22 23:56:12.587673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:12.587673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:12.587673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:12.587673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:12.587673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:12.587673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:12.587673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:12.587673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:12.588817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:12.588817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:12.588817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:12.588817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:12.588817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:12.588817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:12.588817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:56:12.588817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:56:12.588817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:56:12.588817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:56:12.588817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:12.588817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:12.588817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:12.588817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:12.588817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:12.588817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:12.589717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:12.589717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:12.589717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:12.589717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:12.589717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:12.589717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:12.589717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:12.589717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:12.589717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:12.589717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:12.589717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:56:12.589717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:12.589717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 22 23:56:12.589717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:12.592610 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 295/583] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:56:12 2022] GET /identity/v3/services/network => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:12.592987 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:12.603997 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17897d44-084a-472f-9284-b8a050c9c248 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:12.647297 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba18ab-fe68-4227-8b89-25d4afa53c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:12.648785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba18ab-fe68-4227-8b89-25d4afa53c70 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:12.649071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba18ab-fe68-4227-8b89-25d4afa53c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:12.649351 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba18ab-fe68-4227-8b89-25d4afa53c70 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:12.650056 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ba18ab-fe68-4227-8b89-25d4afa53c70 None admin] RBAC: Adding query filter params (interface=public, service_id=d64eba9d959f48be862c8287f386f186, region_id=RegionOne) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:12.650370 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ba18ab-fe68-4227-8b89-25d4afa53c70 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:12.652935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ba18ab-fe68-4227-8b89-25d4afa53c70 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:12.676370 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 289/584] 10.222.0.13 () {56 vars in 1251 bytes} [Sat Oct 22 23:56:12 2022] GET /identity/v3/endpoints?service_id=d64eba9d959f48be862c8287f386f186&interface=public®ion_id=RegionOne => generated 194 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:12.681189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba18ab-fe68-4227-8b89-25d4afa53c70 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:12.758972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17897d44-084a-472f-9284-b8a050c9c248 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:12.799815 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738762b2-0dd1-4e4f-81dd-f9b06df8e5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:12.800718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738762b2-0dd1-4e4f-81dd-f9b06df8e5bf None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:12.801006 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738762b2-0dd1-4e4f-81dd-f9b06df8e5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:12.801287 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738762b2-0dd1-4e4f-81dd-f9b06df8e5bf None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:12.802402 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738762b2-0dd1-4e4f-81dd-f9b06df8e5bf None admin] RBAC: Adding query filter params (name=network) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:12.802721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738762b2-0dd1-4e4f-81dd-f9b06df8e5bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:12.804084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738762b2-0dd1-4e4f-81dd-f9b06df8e5bf None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:12.809664 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 296/585] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Oct 22 23:56:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:12.815700 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738762b2-0dd1-4e4f-81dd-f9b06df8e5bf None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:12.887101 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba18ab-fe68-4227-8b89-25d4afa53c70 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:12.913114 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:12.925410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eeebc9-2166-477e-831e-ec29a963aed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:12.926037 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eeebc9-2166-477e-831e-ec29a963aed9 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:12.926327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eeebc9-2166-477e-831e-ec29a963aed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:12.926608 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eeebc9-2166-477e-831e-ec29a963aed9 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:12.927287 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eeebc9-2166-477e-831e-ec29a963aed9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:12.928712 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eeebc9-2166-477e-831e-ec29a963aed9 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:12.934309 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 290/586] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Oct 22 23:56:12 2022] GET /identity/v3/services => generated 772 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:12.950249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:12.996155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738762b2-0dd1-4e4f-81dd-f9b06df8e5bf None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:13.049827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef907d78-f8c5-430c-b6b8-5ddf8c8ab8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:13.050472 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef907d78-f8c5-430c-b6b8-5ddf8c8ab8b7 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:13.050756 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef907d78-f8c5-430c-b6b8-5ddf8c8ab8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:13.051024 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef907d78-f8c5-430c-b6b8-5ddf8c8ab8b7 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:13.051748 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef907d78-f8c5-430c-b6b8-5ddf8c8ab8b7 None admin] RBAC: Adding query filter params (type=network) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:13.052049 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef907d78-f8c5-430c-b6b8-5ddf8c8ab8b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:13.053458 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef907d78-f8c5-430c-b6b8-5ddf8c8ab8b7 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:13.060322 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 297/587] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Oct 22 23:56:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:13.067105 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eeebc9-2166-477e-831e-ec29a963aed9 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:13.206738 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eeebc9-2166-477e-831e-ec29a963aed9 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:13.242699 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66bd98-b7ba-495a-bff8-ee377d87e449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:13.243315 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66bd98-b7ba-495a-bff8-ee377d87e449 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:13.243598 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66bd98-b7ba-495a-bff8-ee377d87e449 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:13.243876 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66bd98-b7ba-495a-bff8-ee377d87e449 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:13.244598 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f66bd98-b7ba-495a-bff8-ee377d87e449 None admin] RBAC: Adding query filter params (interface=public, service_id=fef91c1743c64d869a328c8c2bac5329, region_id=RegionOne) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:13.244911 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f66bd98-b7ba-495a-bff8-ee377d87e449 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:13.246273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f66bd98-b7ba-495a-bff8-ee377d87e449 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:13.252109 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 291/588] 10.222.0.13 () {56 vars in 1251 bytes} [Sat Oct 22 23:56:13 2022] GET /identity/v3/endpoints?service_id=fef91c1743c64d869a328c8c2bac5329&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:13.257652 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef907d78-f8c5-430c-b6b8-5ddf8c8ab8b7 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:13.368251 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef907d78-f8c5-430c-b6b8-5ddf8c8ab8b7 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:13.398115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:13.405573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea0d2f-6c36-44d2-9675-7449f61367b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:13.406389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea0d2f-6c36-44d2-9675-7449f61367b4 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:13.406854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea0d2f-6c36-44d2-9675-7449f61367b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:13.407458 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea0d2f-6c36-44d2-9675-7449f61367b4 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:13.408065 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea0d2f-6c36-44d2-9675-7449f61367b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:13.409445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea0d2f-6c36-44d2-9675-7449f61367b4 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:13.418961 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 298/589] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Oct 22 23:56:13 2022] GET /identity/v3/services => generated 772 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:14.089557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece0d2a-4907-4884-b488-f72086711143 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:14.090119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece0d2a-4907-4884-b488-f72086711143 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:14.090119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece0d2a-4907-4884-b488-f72086711143 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:14.091701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 292/590] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:14.097478 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271883f-d198-4824-aab3-cb3f4f98dbb2 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:14.098016 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271883f-d198-4824-aab3-cb3f4f98dbb2 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:14.098016 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271883f-d198-4824-aab3-cb3f4f98dbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:14.125972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-d271883f-d198-4824-aab3-cb3f4f98dbb2 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:14.127415 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d271883f-d198-4824-aab3-cb3f4f98dbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:14.217035 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 299/591] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:14.311075 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454ecd4-edc5-4035-b728-b07f4f17bd08 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:14.311359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454ecd4-edc5-4035-b728-b07f4f17bd08 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:14.311815 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454ecd4-edc5-4035-b728-b07f4f17bd08 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:14.313791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 293/592] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:14.326308 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8601e-c407-4a80-8bb2-9b8a96436694 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:14.326592 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8601e-c407-4a80-8bb2-9b8a96436694 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:14.326868 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8601e-c407-4a80-8bb2-9b8a96436694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:14.349189 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-9ed8601e-c407-4a80-8bb2-9b8a96436694 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:14.350286 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9ed8601e-c407-4a80-8bb2-9b8a96436694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:14.446654 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 300/593] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:14.669258 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454ecd4-edc5-4035-b728-b07f4f17bd08 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:14.740939 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:14.814210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454ecd4-edc5-4035-b728-b07f4f17bd08 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:14.817168 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8601e-c407-4a80-8bb2-9b8a96436694 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:14.861664 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78ea56-5e6f-48b4-82dc-554fbdbb641f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:14.862435 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78ea56-5e6f-48b4-82dc-554fbdbb641f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:14.862714 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78ea56-5e6f-48b4-82dc-554fbdbb641f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:14.863104 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78ea56-5e6f-48b4-82dc-554fbdbb641f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:14.874044 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78ea56-5e6f-48b4-82dc-554fbdbb641f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:56:14.879290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-5c78ea56-5e6f-48b4-82dc-554fbdbb641f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:56:14.879290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:14.879290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:14.879290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:14.879290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:14.879290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:14.879290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:14.879290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:14.879290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:14.879290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:14.879290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:14.879290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:14.879290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:56:14.879290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:56:14.879290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:56:14.879290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:56:14.880313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:56:14.880313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:56:14.880313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:56:14.880313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:56:14.880313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:14.880313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:14.880313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:14.880313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:14.880313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:14.880313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:14.880313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:14.880313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:14.880313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:14.880313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:14.880313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:14.880313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:14.881262 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:14.881262 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:14.881262 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:14.881262 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:14.881262 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:56:14.881262 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:56:14.881262 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:56:14.881262 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:56:14.881262 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:56:14.881262 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:56:14.881262 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:56:14.881262 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:14.881262 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 294/594] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Oct 22 23:56:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:14.882727 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9671b-4eca-44e0-9848-c5c15063df04 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:14.883175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9671b-4eca-44e0-9848-c5c15063df04 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:14.883453 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9671b-4eca-44e0-9848-c5c15063df04 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:14.888544 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 295/595] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:14.888544 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9671b-4eca-44e0-9848-c5c15063df04 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:14.896719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:14.953559 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8601e-c407-4a80-8bb2-9b8a96436694 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:14.992030 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1264a-4c73-4737-b6a9-ee5d42d0901c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:14.993022 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1264a-4c73-4737-b6a9-ee5d42d0901c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:14.993516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1264a-4c73-4737-b6a9-ee5d42d0901c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:14.993983 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1264a-4c73-4737-b6a9-ee5d42d0901c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:14.994827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1264a-4c73-4737-b6a9-ee5d42d0901c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:14.996246 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1264a-4c73-4737-b6a9-ee5d42d0901c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:15.005556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-4ab1264a-4c73-4737-b6a9-ee5d42d0901c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 22 23:56:15.005556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:15.005556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:56:15.005556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:56:15.005556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:56:15.005556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:56:15.005556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:56:15.005556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:15.005556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 22 23:56:15.005556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:15.005556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:15.005556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:15.005556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:15.005556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:15.005556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:15.005556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:15.006549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:15.006549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:15.006549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:15.006549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:15.006549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:15.006549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:15.006549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:15.006549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:56:15.006549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:56:15.006549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:56:15.006549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:56:15.006549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:15.006549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:15.006549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:15.006549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:15.006549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:15.006549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:15.007464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:15.007464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:15.007464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:15.007464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:15.007464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:15.007464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:15.007464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:15.007464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:15.007464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:15.007464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:15.007464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:56:15.007464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:15.007464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 22 23:56:15.007464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:15.007464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 301/596] 10.222.0.13 () {56 vars in 1132 bytes} [Sat Oct 22 23:56:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:56:15.008273 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc56bc5-3177-4f06-ba0f-d0a9063b6791 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:15.008898 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc56bc5-3177-4f06-ba0f-d0a9063b6791 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:15.009006 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc56bc5-3177-4f06-ba0f-d0a9063b6791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:15.019559 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9671b-4eca-44e0-9848-c5c15063df04 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:15.028421 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-9bc56bc5-3177-4f06-ba0f-d0a9063b6791 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:15.029720 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9bc56bc5-3177-4f06-ba0f-d0a9063b6791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:15.031926 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:15.069536 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc25ee1-d93c-45e3-9d07-751bffec58fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:15.070191 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc25ee1-d93c-45e3-9d07-751bffec58fa None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:15.070598 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc25ee1-d93c-45e3-9d07-751bffec58fa None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:15.070988 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc25ee1-d93c-45e3-9d07-751bffec58fa None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:15.071749 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc25ee1-d93c-45e3-9d07-751bffec58fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:15.072157 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc25ee1-d93c-45e3-9d07-751bffec58fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:15.073590 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc25ee1-d93c-45e3-9d07-751bffec58fa None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:15.083293 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 296/597] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Oct 22 23:56:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:15.084113 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc25ee1-d93c-45e3-9d07-751bffec58fa None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:15.132019 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 302/598] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:15.132999 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc56bc5-3177-4f06-ba0f-d0a9063b6791 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:15.196415 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:15.230508 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc25ee1-d93c-45e3-9d07-751bffec58fa None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:15.259626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc56bc5-3177-4f06-ba0f-d0a9063b6791 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:15.269318 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d90e9-d1a7-47c9-ae2f-7da259e60ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:15.269925 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d90e9-d1a7-47c9-ae2f-7da259e60ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:15.270204 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d90e9-d1a7-47c9-ae2f-7da259e60ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:15.270473 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d90e9-d1a7-47c9-ae2f-7da259e60ac2 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:15.271515 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d90e9-d1a7-47c9-ae2f-7da259e60ac2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:15.271645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d90e9-d1a7-47c9-ae2f-7da259e60ac2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:15.273287 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d90e9-d1a7-47c9-ae2f-7da259e60ac2 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:15.278459 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 297/599] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Oct 22 23:56:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:15.285100 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d90e9-d1a7-47c9-ae2f-7da259e60ac2 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:15.298593 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4a47f-baf6-4124-be77-cae7865d2c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:15.299702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4a47f-baf6-4124-be77-cae7865d2c7d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:15.300218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4a47f-baf6-4124-be77-cae7865d2c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:15.300731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4a47f-baf6-4124-be77-cae7865d2c7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:15.311528 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4a47f-baf6-4124-be77-cae7865d2c7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:15.313264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4a47f-baf6-4124-be77-cae7865d2c7d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:15.324505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:15.330441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-e3a4a47f-baf6-4124-be77-cae7865d2c7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:15.330441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:15.330441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:15.330441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:15.330441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:15.330441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:15.330441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:15.330441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:15.330441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:15.330441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:15.330441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:15.330441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:15.330441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:15.330441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:15.330441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:15.330441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:15.331420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:15.331420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:15.331420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:15.331420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:15.331420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:15.331420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:15.331420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:15.331420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:15.331420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:15.331420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:15.331420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:15.331420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:15.331420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:15.331420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:15.331420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:15.331420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:15.331420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:15.332359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:15.332359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:15.332359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:15.334454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 303/600] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:15.340108 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4a47f-baf6-4124-be77-cae7865d2c7d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:15.403896 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d90e9-d1a7-47c9-ae2f-7da259e60ac2 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:15.447266 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdebf249-ee6d-472b-8bf2-5f1b4c2f95a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:15.447871 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebf249-ee6d-472b-8bf2-5f1b4c2f95a0 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:15.448148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebf249-ee6d-472b-8bf2-5f1b4c2f95a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:15.448594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebf249-ee6d-472b-8bf2-5f1b4c2f95a0 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:15.449253 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdebf249-ee6d-472b-8bf2-5f1b4c2f95a0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:15.449552 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdebf249-ee6d-472b-8bf2-5f1b4c2f95a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:15.451200 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdebf249-ee6d-472b-8bf2-5f1b4c2f95a0 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:15.457700 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 298/601] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Oct 22 23:56:15 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:15.466769 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:15.472642 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4a47f-baf6-4124-be77-cae7865d2c7d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:15.511879 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780c4e2-5868-426b-9fa3-60edf71ba238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:15.513015 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780c4e2-5868-426b-9fa3-60edf71ba238 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:15.513512 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780c4e2-5868-426b-9fa3-60edf71ba238 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:15.514028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780c4e2-5868-426b-9fa3-60edf71ba238 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:15.514934 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780c4e2-5868-426b-9fa3-60edf71ba238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:15.515456 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780c4e2-5868-426b-9fa3-60edf71ba238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:15.517755 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780c4e2-5868-426b-9fa3-60edf71ba238 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:15.532210 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 304/602] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:15.538245 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdebf249-ee6d-472b-8bf2-5f1b4c2f95a0 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:15.553962 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780c4e2-5868-426b-9fa3-60edf71ba238 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:15.724234 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:15.726101 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780c4e2-5868-426b-9fa3-60edf71ba238 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:15.731410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdebf249-ee6d-472b-8bf2-5f1b4c2f95a0 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:15.773280 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f084e8-5bae-4bce-b901-24803e7f86ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:15.773827 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f084e8-5bae-4bce-b901-24803e7f86ab None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:15.774103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f084e8-5bae-4bce-b901-24803e7f86ab None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:15.774374 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f084e8-5bae-4bce-b901-24803e7f86ab None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:15.782122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f084e8-5bae-4bce-b901-24803e7f86ab None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:15.783619 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f084e8-5bae-4bce-b901-24803e7f86ab None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:15.784460 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f864104-e63f-4e99-9355-5cc0c3dbffd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:15.785635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f864104-e63f-4e99-9355-5cc0c3dbffd8 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:15.786743 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f864104-e63f-4e99-9355-5cc0c3dbffd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:15.787229 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f864104-e63f-4e99-9355-5cc0c3dbffd8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:15.795180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-f8f084e8-5bae-4bce-b901-24803e7f86ab None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 22 23:56:15.795180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:15.795180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:15.795180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:15.795180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:15.795180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:15.795180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:15.795180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:15.795180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:15.795180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:15.795180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:15.795180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:15.795180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:56:15.795180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:56:15.795180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:56:15.795180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:56:15.796161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:15.796161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:15.796161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:56:15.796161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:15.796161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:56:15.796161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:15.796161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:15.796161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:15.796161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:15.796161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:15.796161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:15.796161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:15.796161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:15.796161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:15.796161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:15.796161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:15.796161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:15.796161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:15.797194 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:15.797194 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:15.797194 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:56:15.797194 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:56:15.797194 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:56:15.797194 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:56:15.797194 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:56:15.797194 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:56:15.797194 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 22 23:56:15.797194 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:15.797194 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 299/603] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Oct 22 23:56:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:15.804536 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f084e8-5bae-4bce-b901-24803e7f86ab None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:15.807293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f864104-e63f-4e99-9355-5cc0c3dbffd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:56:15.812082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-1f864104-e63f-4e99-9355-5cc0c3dbffd8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:56:15.812082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:15.812082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:15.812082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:15.812082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:15.812082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:15.812082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:15.812082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:15.812082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:15.812082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:15.812082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:15.812082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:15.812082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:56:15.812082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:56:15.812082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:56:15.812082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:56:15.813073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:56:15.813073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:56:15.813073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:56:15.813073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:56:15.813073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:15.813073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:15.813073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:15.813073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:15.813073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:15.813073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:15.813073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:15.813073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:15.813073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:15.813073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:15.813073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:15.813073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:15.814045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:15.814045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:15.814045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:15.814045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:15.814045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:56:15.814045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:56:15.814045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:56:15.814045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:56:15.814045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:56:15.814045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:56:15.814045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:56:15.814045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:15.823556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 305/604] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Oct 22 23:56:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:56:15.824473 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f864104-e63f-4e99-9355-5cc0c3dbffd8 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:15.858582 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:15.945972 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f084e8-5bae-4bce-b901-24803e7f86ab None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:15.953886 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f864104-e63f-4e99-9355-5cc0c3dbffd8 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:15.991836 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902e3fa-e960-49b3-9715-5f4f7895b338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:15.992049 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb3fe7-aacc-4f58-8af2-9d7aacba81cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:15.992450 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902e3fa-e960-49b3-9715-5f4f7895b338 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:15.992689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb3fe7-aacc-4f58-8af2-9d7aacba81cd None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:15.992805 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902e3fa-e960-49b3-9715-5f4f7895b338 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:15.992969 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb3fe7-aacc-4f58-8af2-9d7aacba81cd None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:15.993063 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902e3fa-e960-49b3-9715-5f4f7895b338 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:15.993230 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb3fe7-aacc-4f58-8af2-9d7aacba81cd None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:15.993732 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902e3fa-e960-49b3-9715-5f4f7895b338 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:15.993916 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb3fe7-aacc-4f58-8af2-9d7aacba81cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:15.994016 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902e3fa-e960-49b3-9715-5f4f7895b338 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:15.994169 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb3fe7-aacc-4f58-8af2-9d7aacba81cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:15.995448 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902e3fa-e960-49b3-9715-5f4f7895b338 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:15.995573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb3fe7-aacc-4f58-8af2-9d7aacba81cd None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:16.009743 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 306/605] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:56:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:16.011078 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63470370-f2c3-4925-b4d4-81c3eceff188 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:16.011364 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63470370-f2c3-4925-b4d4-81c3eceff188 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:16.011642 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63470370-f2c3-4925-b4d4-81c3eceff188 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:16.013252 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 307/606] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:16.013957 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63470370-f2c3-4925-b4d4-81c3eceff188 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:16.023725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 300/607] 10.222.0.13 () {56 vars in 1149 bytes} [Sat Oct 22 23:56:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:16.025836 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f6656-a486-4612-95b4-0158ace57c0f None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:16.026127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f6656-a486-4612-95b4-0158ace57c0f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:16.026435 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f6656-a486-4612-95b4-0158ace57c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:16.051961 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-c81f6656-a486-4612-95b4-0158ace57c0f None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:16.053337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c81f6656-a486-4612-95b4-0158ace57c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:16.068089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:16.119825 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:16.140517 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 301/608] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 22 23:56:16.140517 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f6656-a486-4612-95b4-0158ace57c0f None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:16.164904 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63470370-f2c3-4925-b4d4-81c3eceff188 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:16.221724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670235f-8c4a-4ede-b4e8-be6b42afb9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:16.222740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670235f-8c4a-4ede-b4e8-be6b42afb9e2 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:16.223440 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670235f-8c4a-4ede-b4e8-be6b42afb9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:16.223916 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670235f-8c4a-4ede-b4e8-be6b42afb9e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:16.235316 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4670235f-8c4a-4ede-b4e8-be6b42afb9e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:16.237799 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4670235f-8c4a-4ede-b4e8-be6b42afb9e2 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:16.254632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-4670235f-8c4a-4ede-b4e8-be6b42afb9e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:16.254632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:16.254632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:16.254632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:16.254632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:16.254632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:16.254632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:16.254632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:16.254632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:16.254632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:16.254632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:16.254632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:16.254632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:16.254632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:16.254632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:16.254632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:16.255749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:16.255749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:16.255749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:16.255749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:16.255749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:16.255749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:16.255749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:16.255749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:16.255749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:16.255749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:16.255749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:16.255749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:16.255749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:16.255749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:16.255749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:16.255749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:16.255749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:16.256701 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:16.256701 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:16.256701 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:16.259078 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 308/609] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:16.273649 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670235f-8c4a-4ede-b4e8-be6b42afb9e2 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:16.334829 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f6656-a486-4612-95b4-0158ace57c0f None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:16.382273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12568eee-6c1c-4877-a79b-cd1fe9624c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:16.382913 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12568eee-6c1c-4877-a79b-cd1fe9624c57 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:16.383194 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12568eee-6c1c-4877-a79b-cd1fe9624c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:16.383475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12568eee-6c1c-4877-a79b-cd1fe9624c57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:16.387828 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670235f-8c4a-4ede-b4e8-be6b42afb9e2 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:16.407645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:16.420834 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:16.427756 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780997c1-93ac-42c6-a548-d564689bbb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:16.428710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780997c1-93ac-42c6-a548-d564689bbb4d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:16.429260 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780997c1-93ac-42c6-a548-d564689bbb4d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:16.429732 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780997c1-93ac-42c6-a548-d564689bbb4d None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:16.430554 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780997c1-93ac-42c6-a548-d564689bbb4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:16.431050 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780997c1-93ac-42c6-a548-d564689bbb4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:16.432701 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780997c1-93ac-42c6-a548-d564689bbb4d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:16.446493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 309/610] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:16.448058 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c313a45-4aec-4fd6-a9c1-b16a60e3afd1 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:16.448630 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c313a45-4aec-4fd6-a9c1-b16a60e3afd1 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:16.449135 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c313a45-4aec-4fd6-a9c1-b16a60e3afd1 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:16.456143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 310/611] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:16.457246 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c313a45-4aec-4fd6-a9c1-b16a60e3afd1 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:16.469810 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12568eee-6c1c-4877-a79b-cd1fe9624c57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:16.471489 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12568eee-6c1c-4877-a79b-cd1fe9624c57 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:16.573753 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c313a45-4aec-4fd6-a9c1-b16a60e3afd1 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:16.592935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 302/612] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:56:16 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 22 23:56:16.594316 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e7be0-369e-45ab-adb1-a4722a338533 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:16.594604 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e7be0-369e-45ab-adb1-a4722a338533 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:16.595494 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e7be0-369e-45ab-adb1-a4722a338533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:16.611659 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ec5da-ec0b-4b4b-9040-631df6ca5cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:16.612278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ec5da-ec0b-4b4b-9040-631df6ca5cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:16.612587 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ec5da-ec0b-4b4b-9040-631df6ca5cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:16.612862 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ec5da-ec0b-4b4b-9040-631df6ca5cf1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:16.612963 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-c03e7be0-369e-45ab-adb1-a4722a338533 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:16.614219 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c03e7be0-369e-45ab-adb1-a4722a338533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:16.619956 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ec5da-ec0b-4b4b-9040-631df6ca5cf1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:16.621381 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ec5da-ec0b-4b4b-9040-631df6ca5cf1 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:16.633259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-ac9ec5da-ec0b-4b4b-9040-631df6ca5cf1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 22 23:56:16.633259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:16.633259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:16.633259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:16.633259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:16.633259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:16.633259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:16.633259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:16.633259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:16.633259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:16.633259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:16.633259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:16.633259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:56:16.633259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:56:16.633259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:56:16.633259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:56:16.634211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:16.634211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:16.634211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:56:16.634211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:16.634211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:56:16.634211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:16.634211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:16.634211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:16.634211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:16.634211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:16.634211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:16.634211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:16.634211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:16.634211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:16.634211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:16.634211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:16.634211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:16.634211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:16.635159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:16.635159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:16.635159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:56:16.635159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:56:16.635159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:56:16.635159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:56:16.635159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:56:16.635159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:56:16.635159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 22 23:56:16.635159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:16.635159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 311/613] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Oct 22 23:56:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:16.639637 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ec5da-ec0b-4b4b-9040-631df6ca5cf1 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:16.662951 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:16.689090 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 303/614] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:16.689090 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e7be0-369e-45ab-adb1-a4722a338533 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:16.699430 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:16.753473 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ec5da-ec0b-4b4b-9040-631df6ca5cf1 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:16.791412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7bab22-d2d0-4c1f-a019-ce3e0be1b132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:16.792354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7bab22-d2d0-4c1f-a019-ce3e0be1b132 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:16.792807 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7bab22-d2d0-4c1f-a019-ce3e0be1b132 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:16.793220 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7bab22-d2d0-4c1f-a019-ce3e0be1b132 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:16.793944 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7bab22-d2d0-4c1f-a019-ce3e0be1b132 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:16.795603 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7bab22-d2d0-4c1f-a019-ce3e0be1b132 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:16.805223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-7f7bab22-d2d0-4c1f-a019-ce3e0be1b132 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 22 23:56:16.805223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:16.805223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:56:16.805223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:56:16.805223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:56:16.805223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:56:16.805223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:56:16.805223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:16.805223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 22 23:56:16.805223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:16.805223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:16.805223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:16.805223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:16.805223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:16.805223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:16.805223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:16.805223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:16.806444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:16.806444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:16.806444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:16.806444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:16.806444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:16.806444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:16.806444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:56:16.806444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:56:16.806444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:56:16.806444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:56:16.806444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:16.806444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:16.806444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:16.806444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:16.806444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:16.806444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:16.807434 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:16.807434 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:16.807434 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:16.807434 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:16.807434 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:16.807434 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:16.807434 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:16.807434 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:16.807434 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:16.807434 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:16.807434 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:56:16.807434 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:16.807434 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 22 23:56:16.807434 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:16.815033 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 312/615] 10.222.0.13 () {56 vars in 1129 bytes} [Sat Oct 22 23:56:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:16.816042 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7bab22-d2d0-4c1f-a019-ce3e0be1b132 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:16.829121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e7be0-369e-45ab-adb1-a4722a338533 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:16.866942 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74642c8-0cd0-47a8-98ba-b5dc5c1f60da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:16.867500 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74642c8-0cd0-47a8-98ba-b5dc5c1f60da None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:16.867780 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74642c8-0cd0-47a8-98ba-b5dc5c1f60da None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:16.868054 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74642c8-0cd0-47a8-98ba-b5dc5c1f60da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:16.878962 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74642c8-0cd0-47a8-98ba-b5dc5c1f60da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:16.880429 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74642c8-0cd0-47a8-98ba-b5dc5c1f60da None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:16.895872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-e74642c8-0cd0-47a8-98ba-b5dc5c1f60da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:16.895872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:16.895872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:16.895872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:16.895872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:16.895872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:16.895872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:16.895872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:16.895872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:16.895872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:16.895872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:16.895872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:16.895872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:16.895872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:16.895872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:16.895872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:16.897086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:16.897086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:16.897086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:16.897086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:16.897086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:16.897086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:16.897086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:16.897086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:16.897086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:16.897086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:16.897086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:16.897086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:16.897086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:16.897086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:16.897086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:16.897086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:16.897086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:16.898124 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:16.898124 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:16.898124 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:16.900608 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 304/616] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:16.901431 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74642c8-0cd0-47a8-98ba-b5dc5c1f60da None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:16.933764 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:16.944813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7bab22-d2d0-4c1f-a019-ce3e0be1b132 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:16.966198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:16.988026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989a8a4-3064-48bf-8845-9d7e5b50e821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:16.988806 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989a8a4-3064-48bf-8845-9d7e5b50e821 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:16.989095 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989a8a4-3064-48bf-8845-9d7e5b50e821 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:16.989372 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989a8a4-3064-48bf-8845-9d7e5b50e821 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:16.990098 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e989a8a4-3064-48bf-8845-9d7e5b50e821 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:16.990824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e989a8a4-3064-48bf-8845-9d7e5b50e821 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:16.992616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e989a8a4-3064-48bf-8845-9d7e5b50e821 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:17.011704 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 313/617] 10.222.0.13 () {56 vars in 1135 bytes} [Sat Oct 22 23:56:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:17.012467 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989a8a4-3064-48bf-8845-9d7e5b50e821 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:17.016141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74642c8-0cd0-47a8-98ba-b5dc5c1f60da None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:17.052141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f487c-5bb6-4a03-a965-7d0546398c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:17.052773 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f487c-5bb6-4a03-a965-7d0546398c4e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:17.053058 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f487c-5bb6-4a03-a965-7d0546398c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:17.053330 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f487c-5bb6-4a03-a965-7d0546398c4e None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:17.053949 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7f487c-5bb6-4a03-a965-7d0546398c4e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:17.054254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7f487c-5bb6-4a03-a965-7d0546398c4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:17.055610 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7f487c-5bb6-4a03-a965-7d0546398c4e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:17.060077 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 305/618] 10.222.0.13 () {56 vars in 1124 bytes} [Sat Oct 22 23:56:16 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:17.060883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f487c-5bb6-4a03-a965-7d0546398c4e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:17.129978 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989a8a4-3064-48bf-8845-9d7e5b50e821 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:17.162537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:17.166099 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb40102-7071-42fc-b165-91cfce11e040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:17.166702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb40102-7071-42fc-b165-91cfce11e040 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:17.166984 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb40102-7071-42fc-b165-91cfce11e040 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:17.167261 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb40102-7071-42fc-b165-91cfce11e040 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:17.167891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb40102-7071-42fc-b165-91cfce11e040 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:17.168191 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb40102-7071-42fc-b165-91cfce11e040 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:17.169577 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb40102-7071-42fc-b165-91cfce11e040 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:17.171359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f487c-5bb6-4a03-a965-7d0546398c4e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:17.181651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 314/619] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:17.182463 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb40102-7071-42fc-b165-91cfce11e040 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:17.215632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee1c4a-816a-4d76-b13d-8843f341df47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:17.216610 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee1c4a-816a-4d76-b13d-8843f341df47 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:17.217232 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee1c4a-816a-4d76-b13d-8843f341df47 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:17.217686 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee1c4a-816a-4d76-b13d-8843f341df47 None admin] PATH_INFO: `/v3/services/network` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:17.218499 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee1c4a-816a-4d76-b13d-8843f341df47 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:17.220048 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee1c4a-816a-4d76-b13d-8843f341df47 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:17.221145 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:17.230289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-fbee1c4a-816a-4d76-b13d-8843f341df47 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 22 23:56:17.230289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:17.230289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:56:17.230289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:56:17.230289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:56:17.230289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:56:17.230289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:56:17.230289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:17.230289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 22 23:56:17.230289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:17.230289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:17.230289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:17.230289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:17.230289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:17.230289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:17.230289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:17.230289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:17.231540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:17.231540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:17.231540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:17.231540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:17.231540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:17.231540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:17.231540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:56:17.231540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:56:17.231540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:56:17.231540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:56:17.231540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:17.231540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:17.231540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:17.231540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:17.231540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:17.231540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:17.232679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:17.232679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:17.232679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:17.232679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:17.232679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:17.232679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:17.232679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:17.232679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:17.232679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:17.232679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:17.232679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:56:17.232679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:17.232679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 22 23:56:17.232679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:17.236882 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 306/620] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:56:17 2022] GET /identity/v3/services/network => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:17.237753 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee1c4a-816a-4d76-b13d-8843f341df47 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:17.293438 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb40102-7071-42fc-b165-91cfce11e040 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:17.331388 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d0fe0-90ad-480b-a93f-7d70d15484aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:17.331992 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d0fe0-90ad-480b-a93f-7d70d15484aa None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:17.332272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d0fe0-90ad-480b-a93f-7d70d15484aa None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:17.332580 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d0fe0-90ad-480b-a93f-7d70d15484aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:17.345614 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee1c4a-816a-4d76-b13d-8843f341df47 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:17.382935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33697c-8885-4c29-b6d2-0ac1318fcc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:17.383735 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33697c-8885-4c29-b6d2-0ac1318fcc4a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:17.384019 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33697c-8885-4c29-b6d2-0ac1318fcc4a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:17.384297 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33697c-8885-4c29-b6d2-0ac1318fcc4a None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:17.385122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33697c-8885-4c29-b6d2-0ac1318fcc4a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:17.385805 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33697c-8885-4c29-b6d2-0ac1318fcc4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:17.387167 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33697c-8885-4c29-b6d2-0ac1318fcc4a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:17.392575 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 307/621] 10.222.0.13 () {56 vars in 1124 bytes} [Sat Oct 22 23:56:17 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:17.393247 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33697c-8885-4c29-b6d2-0ac1318fcc4a None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:17.412844 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d0fe0-90ad-480b-a93f-7d70d15484aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:17.414294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d0fe0-90ad-480b-a93f-7d70d15484aa None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:17.419767 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:17.449184 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:17.505682 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33697c-8885-4c29-b6d2-0ac1318fcc4a None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:17.524662 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 315/622] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:56:17 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 22 23:56:17.525537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d0fe0-90ad-480b-a93f-7d70d15484aa None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:17.545857 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d5f3c-6aae-4107-903e-b7c1e7f71201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:17.546486 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d5f3c-6aae-4107-903e-b7c1e7f71201 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:17.546766 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d5f3c-6aae-4107-903e-b7c1e7f71201 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:17.547035 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d5f3c-6aae-4107-903e-b7c1e7f71201 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:17.554369 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d5f3c-6aae-4107-903e-b7c1e7f71201 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:17.555788 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d5f3c-6aae-4107-903e-b7c1e7f71201 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:17.567055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-9f6d5f3c-6aae-4107-903e-b7c1e7f71201 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:17.567055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:17.567055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:17.567055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:17.567055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:17.567055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:17.567055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:17.567055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:17.567055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:17.567055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:17.567055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:17.567055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:17.567055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:56:17.567055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:56:17.567055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:56:17.568273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:56:17.568273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:17.568273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:17.568273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:17.568273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:17.568273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:17.568273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:17.568273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:17.568273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:17.568273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:17.568273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:17.568273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:17.568273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:17.568273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:17.568273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:17.568273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:17.568273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:17.569202 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:56:17.569202 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:56:17.569202 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:56:17.569202 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:56:17.569202 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:56:17.569202 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:56:17.569202 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:17.569202 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:17.569202 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 308/623] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:56:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:17.569202 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d5f3c-6aae-4107-903e-b7c1e7f71201 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:17.638113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d0fe0-90ad-480b-a93f-7d70d15484aa None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:17.653958 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:17.674319 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d5f3c-6aae-4107-903e-b7c1e7f71201 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:17.674711 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2395409-ad17-4f83-8ab3-9581442067f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:17.675322 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2395409-ad17-4f83-8ab3-9581442067f6 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:17.675606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2395409-ad17-4f83-8ab3-9581442067f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:17.675880 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2395409-ad17-4f83-8ab3-9581442067f6 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:17.676529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2395409-ad17-4f83-8ab3-9581442067f6 None admin] RBAC: Adding query filter params (name=network) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:17.676841 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2395409-ad17-4f83-8ab3-9581442067f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:17.678187 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2395409-ad17-4f83-8ab3-9581442067f6 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:17.679264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:17.687701 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 316/624] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Oct 22 23:56:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:17.688769 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2395409-ad17-4f83-8ab3-9581442067f6 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:17.712671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59a1a2-c03b-42dd-abe0-af390b0db9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:17.713325 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59a1a2-c03b-42dd-abe0-af390b0db9da None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:17.714004 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59a1a2-c03b-42dd-abe0-af390b0db9da None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:17.714004 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59a1a2-c03b-42dd-abe0-af390b0db9da None admin] PATH_INFO: `/v3/endpoints` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:17.714514 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59a1a2-c03b-42dd-abe0-af390b0db9da None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:17.715917 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59a1a2-c03b-42dd-abe0-af390b0db9da None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:17.740835 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 309/625] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Oct 22 23:56:17 2022] POST /identity/v3/endpoints => generated 346 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:56:17.741675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59a1a2-c03b-42dd-abe0-af390b0db9da None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:17.820612 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2395409-ad17-4f83-8ab3-9581442067f6 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:17.867347 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653148f-0786-4a0d-8e76-a1d4843d5366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:17.867998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653148f-0786-4a0d-8e76-a1d4843d5366 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:17.868286 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653148f-0786-4a0d-8e76-a1d4843d5366 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:17.868594 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653148f-0786-4a0d-8e76-a1d4843d5366 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:17.881407 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59a1a2-c03b-42dd-abe0-af390b0db9da None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:17.882790 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653148f-0786-4a0d-8e76-a1d4843d5366 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:17.884349 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653148f-0786-4a0d-8e76-a1d4843d5366 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:17.901903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-5653148f-0786-4a0d-8e76-a1d4843d5366 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:17.901903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:17.901903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:17.901903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:17.901903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:17.901903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:17.901903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:17.901903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:17.901903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:17.901903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:17.901903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:17.901903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:17.901903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:17.901903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:17.901903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:17.901903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:17.903112 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:17.903112 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:17.903112 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:17.903112 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:17.903112 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:17.903112 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:17.903112 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:17.903112 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:17.903112 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:17.903112 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:17.903112 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:17.903112 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:17.903112 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:17.903112 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:17.903112 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:17.903112 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:17.903112 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:17.904235 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:17.904235 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:17.904235 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:17.910634 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 317/626] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:17.910634 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653148f-0786-4a0d-8e76-a1d4843d5366 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:17.926744 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1394d8-bff7-406a-bec2-f2b13883fa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:17.927519 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1394d8-bff7-406a-bec2-f2b13883fa82 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:17.927669 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1394d8-bff7-406a-bec2-f2b13883fa82 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:17.927941 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1394d8-bff7-406a-bec2-f2b13883fa82 None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:17.929672 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1394d8-bff7-406a-bec2-f2b13883fa82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:17.929995 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1394d8-bff7-406a-bec2-f2b13883fa82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:17.931456 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1394d8-bff7-406a-bec2-f2b13883fa82 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:17.939500 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:17.945404 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 310/627] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Oct 22 23:56:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:17.946736 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf772e-ca61-4e01-b12c-ee8886c8a3b7 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:17.947446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf772e-ca61-4e01-b12c-ee8886c8a3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:17.947917 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf772e-ca61-4e01-b12c-ee8886c8a3b7 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:17.952914 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 311/628] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:17.953791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf772e-ca61-4e01-b12c-ee8886c8a3b7 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:17.966591 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:18.025748 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653148f-0786-4a0d-8e76-a1d4843d5366 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:18.061314 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4e092-870e-4b6a-89b4-be09883cb6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:18.061919 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4e092-870e-4b6a-89b4-be09883cb6f2 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:18.062200 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4e092-870e-4b6a-89b4-be09883cb6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:18.062480 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4e092-870e-4b6a-89b4-be09883cb6f2 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:18.063116 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b4e092-870e-4b6a-89b4-be09883cb6f2 None admin] RBAC: Adding query filter params (type=network) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:18.063401 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b4e092-870e-4b6a-89b4-be09883cb6f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:18.064174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf772e-ca61-4e01-b12c-ee8886c8a3b7 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:18.065640 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b4e092-870e-4b6a-89b4-be09883cb6f2 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:18.070909 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 318/629] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Oct 22 23:56:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:18.071697 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4e092-870e-4b6a-89b4-be09883cb6f2 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:18.109865 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3f759-f0cc-4938-9017-c85043d624d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:18.110634 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3f759-f0cc-4938-9017-c85043d624d1 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:18.111557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3f759-f0cc-4938-9017-c85043d624d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:18.111813 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3f759-f0cc-4938-9017-c85043d624d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:18.112460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3f759-f0cc-4938-9017-c85043d624d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:18.112792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3f759-f0cc-4938-9017-c85043d624d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:18.114171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3f759-f0cc-4938-9017-c85043d624d1 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:18.128530 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 312/630] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:18.128739 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b8631-e442-4d54-b9c3-4ace945eadf9 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:18.128992 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b8631-e442-4d54-b9c3-4ace945eadf9 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:18.129192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b8631-e442-4d54-b9c3-4ace945eadf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:18.147995 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-910b8631-e442-4d54-b9c3-4ace945eadf9 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:18.149204 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-910b8631-e442-4d54-b9c3-4ace945eadf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:18.190896 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4e092-870e-4b6a-89b4-be09883cb6f2 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:18.201791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:18.207650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:18.222895 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 313/631] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:18.223717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b8631-e442-4d54-b9c3-4ace945eadf9 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:18.235687 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2283ce-7563-4575-95ce-d9b3649616a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:18.236559 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2283ce-7563-4575-95ce-d9b3649616a9 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:18.237000 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2283ce-7563-4575-95ce-d9b3649616a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:18.237465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2283ce-7563-4575-95ce-d9b3649616a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:18.238343 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2283ce-7563-4575-95ce-d9b3649616a9 None admin] RBAC: Adding query filter params (role.id=73721a054c3842bba868ab52c1a3f8e0, scope.project.id=5c9246c059754504afee4711d113a3fe, user.id=7c0356072f92451f98b03a2225e22f86) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:18.238927 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2283ce-7563-4575-95ce-d9b3649616a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:18.240344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2283ce-7563-4575-95ce-d9b3649616a9 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:18.246724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 319/632] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:56:18 2022] GET /identity/v3/role_assignments?user.id=7c0356072f92451f98b03a2225e22f86&scope.project.id=5c9246c059754504afee4711d113a3fe&role.id=73721a054c3842bba868ab52c1a3f8e0 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:18.247563 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2283ce-7563-4575-95ce-d9b3649616a9 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:18.425424 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b8631-e442-4d54-b9c3-4ace945eadf9 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:18.442428 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2283ce-7563-4575-95ce-d9b3649616a9 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:18.483338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30192519-36d7-4f8a-b2cf-9d27a1ec9d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:18.484429 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30192519-36d7-4f8a-b2cf-9d27a1ec9d7e None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:18.485108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30192519-36d7-4f8a-b2cf-9d27a1ec9d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:18.486135 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30192519-36d7-4f8a-b2cf-9d27a1ec9d7e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:18.487576 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48065b75-b77f-49e8-b5b9-5d0c0bf05dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:18.487858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30192519-36d7-4f8a-b2cf-9d27a1ec9d7e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:18.488335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48065b75-b77f-49e8-b5b9-5d0c0bf05dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:18.488714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48065b75-b77f-49e8-b5b9-5d0c0bf05dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:18.489056 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48065b75-b77f-49e8-b5b9-5d0c0bf05dd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:18.490154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30192519-36d7-4f8a-b2cf-9d27a1ec9d7e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:18.496618 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48065b75-b77f-49e8-b5b9-5d0c0bf05dd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:18.498999 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48065b75-b77f-49e8-b5b9-5d0c0bf05dd0 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:18.513338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-48065b75-b77f-49e8-b5b9-5d0c0bf05dd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:18.513338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:18.513338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:18.513338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:18.513338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:18.513338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:18.513338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:18.513338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:18.513338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:18.513338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:18.513338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:18.513338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:18.513338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:56:18.513338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:56:18.513338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:56:18.514285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:56:18.514285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:18.514285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:18.514285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:18.514285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:18.514285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:18.514285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:18.514285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:18.514285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:18.514285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:18.514285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:18.514285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:18.514285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:18.514285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:18.514285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:18.514285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:18.514285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:18.515187 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:56:18.515187 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:56:18.515187 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:56:18.515187 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:56:18.515187 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:56:18.515187 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:56:18.515187 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:18.515187 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:18.519021 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 320/633] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:56:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:18.527096 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48065b75-b77f-49e8-b5b9-5d0c0bf05dd0 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:18.527955 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 314/634] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Oct 22 23:56:18 2022] POST /identity/v3/endpoints => generated 337 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:56:18.564907 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30192519-36d7-4f8a-b2cf-9d27a1ec9d7e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:18.572600 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:18.635409 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:18.662650 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48065b75-b77f-49e8-b5b9-5d0c0bf05dd0 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:18.721124 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342e5c3-77be-45d0-976f-2548b3dfe543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:18.722129 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342e5c3-77be-45d0-976f-2548b3dfe543 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:18.722613 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342e5c3-77be-45d0-976f-2548b3dfe543 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:18.723120 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342e5c3-77be-45d0-976f-2548b3dfe543 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:18.724042 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d342e5c3-77be-45d0-976f-2548b3dfe543 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:18.724575 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d342e5c3-77be-45d0-976f-2548b3dfe543 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:18.726221 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d342e5c3-77be-45d0-976f-2548b3dfe543 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:18.730080 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30192519-36d7-4f8a-b2cf-9d27a1ec9d7e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:18.746461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 321/635] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Oct 22 23:56:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:18.753182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342e5c3-77be-45d0-976f-2548b3dfe543 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:18.781159 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f64d16-aa90-49c3-992f-76181360a3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:18.781775 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f64d16-aa90-49c3-992f-76181360a3b2 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:18.782057 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f64d16-aa90-49c3-992f-76181360a3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:18.782365 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f64d16-aa90-49c3-992f-76181360a3b2 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:18.783092 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f64d16-aa90-49c3-992f-76181360a3b2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:18.784447 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f64d16-aa90-49c3-992f-76181360a3b2 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:18.809238 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 315/636] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Oct 22 23:56:18 2022] POST /identity/v3/services => generated 269 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 22 23:56:18.872962 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:18.885384 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342e5c3-77be-45d0-976f-2548b3dfe543 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:18.932369 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d060f0-07ff-4f6a-975d-234e35f0e9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:18.933019 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d060f0-07ff-4f6a-975d-234e35f0e9fe None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:18.933303 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d060f0-07ff-4f6a-975d-234e35f0e9fe None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:18.933577 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d060f0-07ff-4f6a-975d-234e35f0e9fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:18.934300 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d060f0-07ff-4f6a-975d-234e35f0e9fe None admin] RBAC: Adding query filter params (role.id=839a55263e2c4cbaa2f945b75cf298d6, scope.project.id=5c9246c059754504afee4711d113a3fe, user.id=bfd8c7516b7c435d88298672b2596d6d) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:18.934608 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d060f0-07ff-4f6a-975d-234e35f0e9fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:18.936059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d060f0-07ff-4f6a-975d-234e35f0e9fe None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:18.944080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 322/637] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:56:18 2022] GET /identity/v3/role_assignments?user.id=bfd8c7516b7c435d88298672b2596d6d&scope.project.id=5c9246c059754504afee4711d113a3fe&role.id=839a55263e2c4cbaa2f945b75cf298d6 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:20.030402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32adbe-6910-4d18-99dc-9d0165c43989 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:20.030922 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32adbe-6910-4d18-99dc-9d0165c43989 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:20.031039 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32adbe-6910-4d18-99dc-9d0165c43989 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:20.032641 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 316/638] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:20.037968 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e86ee9-fe39-4ea4-bd3a-d9efa8c04dd2 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:20.038355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e86ee9-fe39-4ea4-bd3a-d9efa8c04dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:20.038528 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e86ee9-fe39-4ea4-bd3a-d9efa8c04dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:20.057484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-e9e86ee9-fe39-4ea4-bd3a-d9efa8c04dd2 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:20.058695 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e9e86ee9-fe39-4ea4-bd3a-d9efa8c04dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:20.134364 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 323/639] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:20 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:20.404673 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32adbe-6910-4d18-99dc-9d0165c43989 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:20.514030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32adbe-6910-4d18-99dc-9d0165c43989 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:20.517261 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:20.555257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2807e37-1447-43c9-9e83-9d160a07c789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:20.555890 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2807e37-1447-43c9-9e83-9d160a07c789 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:20.556171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2807e37-1447-43c9-9e83-9d160a07c789 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:20.556440 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2807e37-1447-43c9-9e83-9d160a07c789 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:20.557061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2807e37-1447-43c9-9e83-9d160a07c789 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:20.558417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2807e37-1447-43c9-9e83-9d160a07c789 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:20.568618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-a2807e37-1447-43c9-9e83-9d160a07c789 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 22 23:56:20.568618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:20.568618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:56:20.568618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:56:20.568618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:56:20.568618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:56:20.568618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:56:20.568618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:20.568618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 22 23:56:20.568618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:20.568618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:20.568618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:20.568618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:20.568618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:20.568618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:20.568618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:20.568618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:20.569656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:20.569656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:20.569656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:20.569656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:20.569656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:20.569656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:20.569656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:56:20.569656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:56:20.569656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:56:20.569656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:56:20.569656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:20.569656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:20.569656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:20.569656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:20.569656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:20.569656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:20.570540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:20.570540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:20.570540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:20.570540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:20.570540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:20.570540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:20.570540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:20.570540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:20.570540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:20.570540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:20.570540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:56:20.570540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:20.570540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 22 23:56:20.570540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:20.570540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 317/640] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Oct 22 23:56:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:20.575996 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e86ee9-fe39-4ea4-bd3a-d9efa8c04dd2 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:20.666116 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:20.689447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e86ee9-fe39-4ea4-bd3a-d9efa8c04dd2 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:20.726800 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f81faa8-1893-4b49-be82-d6ecdaa2382b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:20.727408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81faa8-1893-4b49-be82-d6ecdaa2382b None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:20.727687 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81faa8-1893-4b49-be82-d6ecdaa2382b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:20.727962 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81faa8-1893-4b49-be82-d6ecdaa2382b None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:20.728615 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f81faa8-1893-4b49-be82-d6ecdaa2382b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:20.728918 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f81faa8-1893-4b49-be82-d6ecdaa2382b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:20.730318 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f81faa8-1893-4b49-be82-d6ecdaa2382b None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:20.735938 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 324/641] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Oct 22 23:56:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:20.741537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2807e37-1447-43c9-9e83-9d160a07c789 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:20.781547 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd2506-f8b8-41c7-a7c4-cda7bcc5a788 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:20.781858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd2506-f8b8-41c7-a7c4-cda7bcc5a788 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:20.782089 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd2506-f8b8-41c7-a7c4-cda7bcc5a788 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:20.783936 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 325/642] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:20.789087 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcf862-eb6d-4e84-821a-79b288f812dc None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:20.789372 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcf862-eb6d-4e84-821a-79b288f812dc None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:20.789655 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcf862-eb6d-4e84-821a-79b288f812dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:20.808711 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-9afcf862-eb6d-4e84-821a-79b288f812dc None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:20.809901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9afcf862-eb6d-4e84-821a-79b288f812dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:20.865564 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2807e37-1447-43c9-9e83-9d160a07c789 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:20.895006 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 326/643] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:20 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 22 23:56:20.909864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5780aac-9647-443a-8f52-e196b79be17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:20.910843 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5780aac-9647-443a-8f52-e196b79be17c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:20.911334 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5780aac-9647-443a-8f52-e196b79be17c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:20.911857 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5780aac-9647-443a-8f52-e196b79be17c None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:20.912731 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5780aac-9647-443a-8f52-e196b79be17c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:20.913304 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5780aac-9647-443a-8f52-e196b79be17c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:20.914948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5780aac-9647-443a-8f52-e196b79be17c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:20.920128 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 318/644] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Oct 22 23:56:20 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:21.147654 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44655a-f601-4bd1-a081-ffffde9b720f None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:21.148313 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44655a-f601-4bd1-a081-ffffde9b720f None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:21.148313 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec44655a-f601-4bd1-a081-ffffde9b720f None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:21.149702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 327/645] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:21.155154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928ddec-a731-4cf3-afe3-9910169e9970 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:21.155630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928ddec-a731-4cf3-afe3-9910169e9970 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:21.155789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928ddec-a731-4cf3-afe3-9910169e9970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:21.167012 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44655a-f601-4bd1-a081-ffffde9b720f None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:21.176872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-5928ddec-a731-4cf3-afe3-9910169e9970 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:21.178592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-5928ddec-a731-4cf3-afe3-9910169e9970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:21.214144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:21.253807 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:21.264098 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 319/646] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:21 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:21.265515 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47182ce9-91ee-4b28-8b23-ef0114045e4a None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:21.265800 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47182ce9-91ee-4b28-8b23-ef0114045e4a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:21.266072 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47182ce9-91ee-4b28-8b23-ef0114045e4a None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:21.271683 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 320/647] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:21.272871 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c95de-fa38-4268-9114-894c5467dbb7 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:21.273155 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c95de-fa38-4268-9114-894c5467dbb7 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:21.273429 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c95de-fa38-4268-9114-894c5467dbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:21.284921 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec44655a-f601-4bd1-a081-ffffde9b720f None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:21.292773 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-c18c95de-fa38-4268-9114-894c5467dbb7 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:21.294160 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c18c95de-fa38-4268-9114-894c5467dbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:21.325392 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d246df-6eff-49fd-9591-08d9272b9280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:21.326049 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d246df-6eff-49fd-9591-08d9272b9280 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:21.326383 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d246df-6eff-49fd-9591-08d9272b9280 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:21.326685 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d246df-6eff-49fd-9591-08d9272b9280 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:21.339984 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d246df-6eff-49fd-9591-08d9272b9280 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:56:21.346543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-e5d246df-6eff-49fd-9591-08d9272b9280 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 22 23:56:21.346543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:21.346543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:21.346543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:21.346543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:21.346543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:21.346543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:21.346543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:21.346543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:21.346543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:21.346543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:21.346543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:21.346543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:56:21.346543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:56:21.346543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:56:21.346543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:56:21.347524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:56:21.347524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:56:21.347524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:56:21.347524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:56:21.347524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:21.347524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:21.347524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:21.347524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:21.347524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:21.347524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:21.347524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:21.347524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:21.347524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:21.347524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:21.347524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:21.347524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:21.348403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:21.348403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:21.348403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:21.348403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:21.348403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:56:21.348403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:56:21.348403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:56:21.348403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:56:21.348403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:56:21.348403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:56:21.348403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 22 23:56:21.348403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:21.348403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 328/648] 10.222.0.13 () {56 vars in 1120 bytes} [Sat Oct 22 23:56:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:21.353442 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d246df-6eff-49fd-9591-08d9272b9280 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:21.369799 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 321/649] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:21 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:21.463454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d246df-6eff-49fd-9591-08d9272b9280 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:21.499770 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48e442-26fb-4208-b7c7-29fa5b249445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:21.500802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48e442-26fb-4208-b7c7-29fa5b249445 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:21.501474 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48e442-26fb-4208-b7c7-29fa5b249445 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:21.501961 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48e442-26fb-4208-b7c7-29fa5b249445 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:21.502850 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b48e442-26fb-4208-b7c7-29fa5b249445 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:21.503360 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b48e442-26fb-4208-b7c7-29fa5b249445 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:21.505004 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b48e442-26fb-4208-b7c7-29fa5b249445 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:21.506545 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:21.515277 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 329/650] 10.222.0.13 () {56 vars in 1115 bytes} [Sat Oct 22 23:56:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:21.528477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c95de-fa38-4268-9114-894c5467dbb7 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:21.544585 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48e442-26fb-4208-b7c7-29fa5b249445 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:21.618600 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:21.628308 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c95de-fa38-4268-9114-894c5467dbb7 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:21.666186 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4596cb4-9bbd-4eb3-a29a-d06d7347228b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:21.666819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4596cb4-9bbd-4eb3-a29a-d06d7347228b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:21.667102 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4596cb4-9bbd-4eb3-a29a-d06d7347228b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:21.667379 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4596cb4-9bbd-4eb3-a29a-d06d7347228b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:21.678676 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4596cb4-9bbd-4eb3-a29a-d06d7347228b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:21.679063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48e442-26fb-4208-b7c7-29fa5b249445 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:21.680117 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4596cb4-9bbd-4eb3-a29a-d06d7347228b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:21.695503 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-f4596cb4-9bbd-4eb3-a29a-d06d7347228b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:21.695503 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:21.695503 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:21.695503 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:21.695503 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:21.695503 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:21.695503 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:21.695503 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:21.695503 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:21.695503 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:21.695503 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:21.695503 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:21.695503 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:21.695503 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:21.695503 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:21.695503 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:21.696541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:21.696541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:21.696541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:21.696541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:21.696541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:21.696541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:21.696541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:21.696541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:21.696541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:21.696541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:21.696541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:21.696541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:21.696541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:21.696541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:21.696541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:21.696541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:21.696541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:21.697502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:21.697502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:21.697502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:21.697502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 322/651] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:21.698304 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4596cb4-9bbd-4eb3-a29a-d06d7347228b None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:21.720908 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450360de-b1cc-444e-894e-03649cbc0fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:21.721640 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450360de-b1cc-444e-894e-03649cbc0fcd None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:21.721924 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450360de-b1cc-444e-894e-03649cbc0fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:21.722194 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450360de-b1cc-444e-894e-03649cbc0fcd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:21.722810 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450360de-b1cc-444e-894e-03649cbc0fcd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:21.724277 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450360de-b1cc-444e-894e-03649cbc0fcd None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:21.733395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-450360de-b1cc-444e-894e-03649cbc0fcd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 22 23:56:21.733395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:21.733395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:56:21.733395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:56:21.733395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:56:21.733395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:56:21.733395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:56:21.733395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:21.733395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 22 23:56:21.733395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:21.733395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:21.733395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:21.733395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:21.733395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:21.733395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:21.733395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:21.734354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:21.734354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:21.734354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:21.734354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:21.734354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:21.734354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:21.734354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:21.734354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:56:21.734354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:56:21.734354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:56:21.734354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:56:21.734354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:21.734354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:21.734354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:21.734354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:21.734354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:21.734354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:21.735264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:21.735264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:21.735264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:21.735264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:21.735264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:21.735264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:21.735264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:21.735264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:21.735264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:21.735264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:21.735264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:56:21.735264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:21.735264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 22 23:56:21.735264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:21.735264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 330/652] 10.222.0.13 () {56 vars in 1132 bytes} [Sat Oct 22 23:56:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:56:21.736061 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450360de-b1cc-444e-894e-03649cbc0fcd None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:21.799421 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:21.809458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4596cb4-9bbd-4eb3-a29a-d06d7347228b None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:21.844199 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42dd6c5-2218-4ced-8480-4459fa820c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:21.845428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42dd6c5-2218-4ced-8480-4459fa820c6d None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:21.845710 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42dd6c5-2218-4ced-8480-4459fa820c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:21.846023 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42dd6c5-2218-4ced-8480-4459fa820c6d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:21.846310 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450360de-b1cc-444e-894e-03649cbc0fcd None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:21.846664 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42dd6c5-2218-4ced-8480-4459fa820c6d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:21.848070 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42dd6c5-2218-4ced-8480-4459fa820c6d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:21.857454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-c42dd6c5-2218-4ced-8480-4459fa820c6d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 22 23:56:21.857454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:21.857454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:56:21.857454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:56:21.857454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:56:21.857454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:56:21.857454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:56:21.857454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:21.857454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 22 23:56:21.857454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:21.857454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:21.857454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:21.857454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:21.857454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:21.857454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:21.857454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:21.857454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:21.858596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:21.858596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:21.858596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:21.858596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:21.858596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:21.858596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:21.858596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:56:21.858596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:56:21.858596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:56:21.858596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:56:21.858596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:21.858596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:21.858596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:21.858596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:21.858596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:21.858596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:21.859466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:21.859466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:21.859466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:21.859466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:21.859466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:21.859466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:21.859466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:21.859466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:21.859466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:21.859466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:21.859466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:56:21.859466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:21.859466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 22 23:56:21.859466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:21.859466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 323/653] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Oct 22 23:56:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:21.860278 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42dd6c5-2218-4ced-8480-4459fa820c6d None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:21.873442 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:21.890774 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7c006-9cb4-4f61-840f-5c7485481f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:21.891379 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7c006-9cb4-4f61-840f-5c7485481f5b None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:21.891662 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7c006-9cb4-4f61-840f-5c7485481f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:21.892051 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7c006-9cb4-4f61-840f-5c7485481f5b None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:21.892919 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e7c006-9cb4-4f61-840f-5c7485481f5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:21.893555 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e7c006-9cb4-4f61-840f-5c7485481f5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:21.895186 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e7c006-9cb4-4f61-840f-5c7485481f5b None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:21.909212 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 331/654] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:21.910354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7c006-9cb4-4f61-840f-5c7485481f5b None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:21.971232 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42dd6c5-2218-4ced-8480-4459fa820c6d None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:22.004977 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9c600-3d87-4604-ad61-b9affb5190b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:22.005576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9c600-3d87-4604-ad61-b9affb5190b4 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:22.005857 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9c600-3d87-4604-ad61-b9affb5190b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:22.006127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9c600-3d87-4604-ad61-b9affb5190b4 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:22.006743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc9c600-3d87-4604-ad61-b9affb5190b4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:22.007041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc9c600-3d87-4604-ad61-b9affb5190b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:22.008390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc9c600-3d87-4604-ad61-b9affb5190b4 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:22.012835 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 324/655] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Oct 22 23:56:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:22.018375 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9c600-3d87-4604-ad61-b9affb5190b4 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:22.023422 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7c006-9cb4-4f61-840f-5c7485481f5b None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:22.045512 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:22.062814 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f2c17-6a9b-43f8-9959-b035d5ae5d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:22.063426 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f2c17-6a9b-43f8-9959-b035d5ae5d04 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:22.064231 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f2c17-6a9b-43f8-9959-b035d5ae5d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:22.064534 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f2c17-6a9b-43f8-9959-b035d5ae5d04 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:22.065174 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f2c17-6a9b-43f8-9959-b035d5ae5d04 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:22.065480 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f2c17-6a9b-43f8-9959-b035d5ae5d04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:22.067547 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f2c17-6a9b-43f8-9959-b035d5ae5d04 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:22.077013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 332/656] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Oct 22 23:56:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:22.077013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f2c17-6a9b-43f8-9959-b035d5ae5d04 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:22.115741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:22.140361 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9c600-3d87-4604-ad61-b9affb5190b4 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:22.177628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aca88a-d645-4853-9b24-eb66638a564d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:22.178371 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aca88a-d645-4853-9b24-eb66638a564d None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:22.178559 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aca88a-d645-4853-9b24-eb66638a564d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:22.178863 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aca88a-d645-4853-9b24-eb66638a564d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:22.186547 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aca88a-d645-4853-9b24-eb66638a564d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:22.187975 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aca88a-d645-4853-9b24-eb66638a564d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:22.189330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f2c17-6a9b-43f8-9959-b035d5ae5d04 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:22.199822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-d3aca88a-d645-4853-9b24-eb66638a564d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 22 23:56:22.199822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:22.199822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:22.199822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:22.199822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:22.199822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:22.199822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:22.199822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:22.199822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:22.199822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:22.199822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:22.199822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:22.199822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:56:22.199822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:56:22.199822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:56:22.199822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:56:22.200806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:22.200806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:22.200806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:56:22.200806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:22.200806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:56:22.200806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:22.200806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:22.200806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:22.200806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:22.200806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:22.200806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:22.200806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:22.200806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:22.200806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:22.200806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:22.200806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:22.200806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:22.200806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:22.201776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:22.201776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:22.201776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:56:22.201776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:56:22.201776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:56:22.201776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:56:22.201776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:56:22.201776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:56:22.201776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 22 23:56:22.201776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:22.201776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 325/657] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Oct 22 23:56:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:22.205847 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aca88a-d645-4853-9b24-eb66638a564d None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:22.232315 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed880e4-5e31-4c92-8e3f-805e8ab7b80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:22.233048 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed880e4-5e31-4c92-8e3f-805e8ab7b80d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:22.233329 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed880e4-5e31-4c92-8e3f-805e8ab7b80d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:22.233600 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed880e4-5e31-4c92-8e3f-805e8ab7b80d None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:22.234220 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed880e4-5e31-4c92-8e3f-805e8ab7b80d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:22.234522 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed880e4-5e31-4c92-8e3f-805e8ab7b80d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:22.237124 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed880e4-5e31-4c92-8e3f-805e8ab7b80d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:22.246744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 333/658] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Oct 22 23:56:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:22.246744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed880e4-5e31-4c92-8e3f-805e8ab7b80d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:22.297625 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:22.319509 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aca88a-d645-4853-9b24-eb66638a564d None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:22.347955 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:22.353525 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eecbf0-d126-401a-b048-77848746566c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:22.354125 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eecbf0-d126-401a-b048-77848746566c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:22.354356 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed880e4-5e31-4c92-8e3f-805e8ab7b80d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:22.354464 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eecbf0-d126-401a-b048-77848746566c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:22.354647 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eecbf0-d126-401a-b048-77848746566c None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:22.355271 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eecbf0-d126-401a-b048-77848746566c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:22.355573 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eecbf0-d126-401a-b048-77848746566c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:22.357041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eecbf0-d126-401a-b048-77848746566c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:22.362546 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 326/659] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Oct 22 23:56:22 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:22.363339 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eecbf0-d126-401a-b048-77848746566c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:22.408227 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca7b27-31a4-4e05-b6c0-649ea83ec8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:22.409262 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca7b27-31a4-4e05-b6c0-649ea83ec8e8 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:22.410215 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca7b27-31a4-4e05-b6c0-649ea83ec8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:22.410962 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca7b27-31a4-4e05-b6c0-649ea83ec8e8 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:22.412040 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca7b27-31a4-4e05-b6c0-649ea83ec8e8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:22.412674 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca7b27-31a4-4e05-b6c0-649ea83ec8e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:22.414206 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca7b27-31a4-4e05-b6c0-649ea83ec8e8 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:22.439696 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 334/660] 10.222.0.13 () {56 vars in 1149 bytes} [Sat Oct 22 23:56:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:22.444901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca7b27-31a4-4e05-b6c0-649ea83ec8e8 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:22.518284 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eecbf0-d126-401a-b048-77848746566c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:22.568925 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f5483-cf2a-4d6f-a759-9becdcf3e92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:22.569543 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f5483-cf2a-4d6f-a759-9becdcf3e92c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:22.569806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f5483-cf2a-4d6f-a759-9becdcf3e92c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:22.570077 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f5483-cf2a-4d6f-a759-9becdcf3e92c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:22.570869 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f5483-cf2a-4d6f-a759-9becdcf3e92c None admin] RBAC: Adding query filter params (interface=public, service_id=4fbb0d22ad2444d18c30597a7cb8c836, region_id=RegionOne) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:22.571068 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f5483-cf2a-4d6f-a759-9becdcf3e92c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:22.572418 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f5483-cf2a-4d6f-a759-9becdcf3e92c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:22.575026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca7b27-31a4-4e05-b6c0-649ea83ec8e8 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:22.579134 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 327/661] 10.222.0.13 () {56 vars in 1251 bytes} [Sat Oct 22 23:56:22 2022] GET /identity/v3/endpoints?service_id=4fbb0d22ad2444d18c30597a7cb8c836&interface=public®ion_id=RegionOne => generated 194 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:22.585359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:22.586346 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f5483-cf2a-4d6f-a759-9becdcf3e92c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:22.624395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd173cb-a757-4b44-9307-2e2671b9c4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:22.625167 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd173cb-a757-4b44-9307-2e2671b9c4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:22.625605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd173cb-a757-4b44-9307-2e2671b9c4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:22.625976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd173cb-a757-4b44-9307-2e2671b9c4b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:22.642974 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd173cb-a757-4b44-9307-2e2671b9c4b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:22.645662 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd173cb-a757-4b44-9307-2e2671b9c4b5 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:22.662976 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:22.669517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-fcd173cb-a757-4b44-9307-2e2671b9c4b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:22.669517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:22.669517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:22.669517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:22.669517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:22.669517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:22.669517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:22.669517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:22.669517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:22.669517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:22.669517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:22.669517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:22.669517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:22.669517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:22.669517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:22.669517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:22.670803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:22.670803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:22.670803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:22.670803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:22.670803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:22.670803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:22.670803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:22.670803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:22.670803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:22.670803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:22.670803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:22.670803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:22.670803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:22.670803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:22.670803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:22.670803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:22.670803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:22.672021 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:22.672021 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:22.672021 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:22.678397 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 335/662] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:22.680263 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd173cb-a757-4b44-9307-2e2671b9c4b5 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:22.716894 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f5483-cf2a-4d6f-a759-9becdcf3e92c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:22.753246 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14023807-7cba-4d3f-b132-609254e5cb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:22.754222 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14023807-7cba-4d3f-b132-609254e5cb70 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:22.754222 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14023807-7cba-4d3f-b132-609254e5cb70 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:22.754419 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14023807-7cba-4d3f-b132-609254e5cb70 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:22.755017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14023807-7cba-4d3f-b132-609254e5cb70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:22.756373 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14023807-7cba-4d3f-b132-609254e5cb70 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:22.762546 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 328/663] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Oct 22 23:56:22 2022] GET /identity/v3/services => generated 1029 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 22 23:56:22.811520 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd173cb-a757-4b44-9307-2e2671b9c4b5 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:22.851820 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e2588-8212-4321-bbc0-6c9afc3341c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:22.852642 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e2588-8212-4321-bbc0-6c9afc3341c3 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:22.853025 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e2588-8212-4321-bbc0-6c9afc3341c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:22.853400 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e2588-8212-4321-bbc0-6c9afc3341c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:22.854225 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e2588-8212-4321-bbc0-6c9afc3341c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:22.854647 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e2588-8212-4321-bbc0-6c9afc3341c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:22.856140 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e2588-8212-4321-bbc0-6c9afc3341c3 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:22.869942 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 336/664] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:22.875890 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14023807-7cba-4d3f-b132-609254e5cb70 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:22.994705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14023807-7cba-4d3f-b132-609254e5cb70 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:23.019601 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:23.032850 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fc574-6384-4e6a-9644-5f059f8a8b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:23.033641 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fc574-6384-4e6a-9644-5f059f8a8b80 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:23.033926 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fc574-6384-4e6a-9644-5f059f8a8b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:23.034206 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fc574-6384-4e6a-9644-5f059f8a8b80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:23.042201 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2fc574-6384-4e6a-9644-5f059f8a8b80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:23.043679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2fc574-6384-4e6a-9644-5f059f8a8b80 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:23.057401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-1d2fc574-6384-4e6a-9644-5f059f8a8b80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:23.057401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:23.057401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:23.057401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:23.057401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:23.057401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:23.057401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:23.057401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:23.057401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:23.057401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:23.057401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:23.057401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:23.057401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:56:23.057401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:56:23.057401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:56:23.058400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:56:23.058400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:23.058400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:23.058400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:23.058400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:23.058400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:23.058400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:23.058400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:23.058400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:23.058400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:23.058400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:23.058400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:23.058400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:23.058400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:23.058400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:23.058400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:23.058400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:23.059317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:56:23.059317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:56:23.059317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:56:23.059317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:56:23.059317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:56:23.059317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:56:23.059317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:23.059317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:23.059317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 329/665] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:56:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:23.066967 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e2588-8212-4321-bbc0-6c9afc3341c3 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:23.087785 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:23.160052 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a44d7b-a8da-40fc-8954-1bd23f5ab20f None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:23.160406 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a44d7b-a8da-40fc-8954-1bd23f5ab20f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:23.160607 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a44d7b-a8da-40fc-8954-1bd23f5ab20f None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:23.162089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 330/666] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:23.168136 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b281a27-a8bf-422e-b68a-925586781b67 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:23.168136 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b281a27-a8bf-422e-b68a-925586781b67 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:23.168136 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b281a27-a8bf-422e-b68a-925586781b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:23.189910 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-5b281a27-a8bf-422e-b68a-925586781b67 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:23.192748 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-5b281a27-a8bf-422e-b68a-925586781b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:23.194888 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e2588-8212-4321-bbc0-6c9afc3341c3 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:23.239858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ab7aa-0b27-42e5-82f4-676239bf366a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:23.240568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ab7aa-0b27-42e5-82f4-676239bf366a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:23.240946 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ab7aa-0b27-42e5-82f4-676239bf366a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:23.241287 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ab7aa-0b27-42e5-82f4-676239bf366a None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:23.241966 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ab7aa-0b27-42e5-82f4-676239bf366a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:23.242278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ab7aa-0b27-42e5-82f4-676239bf366a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:23.243910 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ab7aa-0b27-42e5-82f4-676239bf366a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:23.259139 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 337/667] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Oct 22 23:56:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:23.272534 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ab7aa-0b27-42e5-82f4-676239bf366a None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:23.279360 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 331/668] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:23 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 22 23:56:23.350972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:23.375179 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ab7aa-0b27-42e5-82f4-676239bf366a None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:23.410787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a015911-c410-4de9-ad0b-b81e4106189a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:23.411446 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a015911-c410-4de9-ad0b-b81e4106189a None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:23.411727 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a015911-c410-4de9-ad0b-b81e4106189a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:23.412003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a015911-c410-4de9-ad0b-b81e4106189a None admin] PATH_INFO: `/v3/projects/5c9246c059754504afee4711d113a3fe/users/7c0356072f92451f98b03a2225e22f86/roles/348c1c2575e44dc7a916de12f69abca6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:23.414218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a015911-c410-4de9-ad0b-b81e4106189a None admin] RBAC: Authorizing `identity:create_grant(project_id=5c9246c059754504afee4711d113a3fe, user_id=7c0356072f92451f98b03a2225e22f86, role_id=348c1c2575e44dc7a916de12f69abca6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:23.415628 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a015911-c410-4de9-ad0b-b81e4106189a None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:23.432290 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 338/669] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:56:23 2022] PUT /identity/v3/projects/5c9246c059754504afee4711d113a3fe/users/7c0356072f92451f98b03a2225e22f86/roles/348c1c2575e44dc7a916de12f69abca6 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 22 23:56:23.557310 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b281a27-a8bf-422e-b68a-925586781b67 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:23.666374 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b281a27-a8bf-422e-b68a-925586781b67 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:23.685564 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:23.710503 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579e348-72ff-4484-a329-c6af92d7cb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:23.711450 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579e348-72ff-4484-a329-c6af92d7cb2b None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:23.711931 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579e348-72ff-4484-a329-c6af92d7cb2b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:23.712387 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579e348-72ff-4484-a329-c6af92d7cb2b None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:23.713259 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7579e348-72ff-4484-a329-c6af92d7cb2b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:23.715281 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7579e348-72ff-4484-a329-c6af92d7cb2b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:23.734634 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 332/670] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Oct 22 23:56:23 2022] POST /identity/v3/services => generated 251 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:56:24.720145 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3feb34-556b-4246-923b-87a10259f1d5 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:24.720965 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3feb34-556b-4246-923b-87a10259f1d5 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:24.721080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3feb34-556b-4246-923b-87a10259f1d5 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:24.723339 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 339/671] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:24.729075 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793e369-a0a6-46cb-8569-7a0c8cfffe42 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:24.730178 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793e369-a0a6-46cb-8569-7a0c8cfffe42 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:24.730922 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793e369-a0a6-46cb-8569-7a0c8cfffe42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:24.753063 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-6793e369-a0a6-46cb-8569-7a0c8cfffe42 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:24.754293 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-6793e369-a0a6-46cb-8569-7a0c8cfffe42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:24.830881 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 333/672] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:25.104463 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3feb34-556b-4246-923b-87a10259f1d5 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:25.124319 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb9f07-e5bb-4ecb-b689-66e9c4030352 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:25.124675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb9f07-e5bb-4ecb-b689-66e9c4030352 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:25.124915 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cb9f07-e5bb-4ecb-b689-66e9c4030352 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:25.127383 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 334/673] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 22 23:56:25.133642 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90017fd5-6e07-41fd-866e-c1b3ef332dce None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:25.133936 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90017fd5-6e07-41fd-866e-c1b3ef332dce None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:25.134329 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90017fd5-6e07-41fd-866e-c1b3ef332dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:25.156319 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-90017fd5-6e07-41fd-866e-c1b3ef332dce None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:25.158552 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-90017fd5-6e07-41fd-866e-c1b3ef332dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:25.223272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3feb34-556b-4246-923b-87a10259f1d5 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:25.252583 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 335/674] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:25 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 22 23:56:25.270156 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8bb6a-34b3-4249-824f-efa86b4c3099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:25.271045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8bb6a-34b3-4249-824f-efa86b4c3099 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:25.271548 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8bb6a-34b3-4249-824f-efa86b4c3099 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:25.272018 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8bb6a-34b3-4249-824f-efa86b4c3099 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:25.272930 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8bb6a-34b3-4249-824f-efa86b4c3099 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:25.274476 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8bb6a-34b3-4249-824f-efa86b4c3099 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:25.286136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:25.293554 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 340/675] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Oct 22 23:56:25 2022] POST /identity/v3/services => generated 241 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:56:25.533199 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90017fd5-6e07-41fd-866e-c1b3ef332dce None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:25.563916 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:25.636799 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90017fd5-6e07-41fd-866e-c1b3ef332dce None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:25.682205 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590fcf5-95c7-40ef-903d-fff02d327bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:25.682858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590fcf5-95c7-40ef-903d-fff02d327bcc None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:25.683135 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590fcf5-95c7-40ef-903d-fff02d327bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:25.683475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590fcf5-95c7-40ef-903d-fff02d327bcc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:25.684077 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590fcf5-95c7-40ef-903d-fff02d327bcc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:25.685508 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590fcf5-95c7-40ef-903d-fff02d327bcc None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:25.696398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-5590fcf5-95c7-40ef-903d-fff02d327bcc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 22 23:56:25.696398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:25.696398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:56:25.696398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:56:25.696398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:56:25.696398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:56:25.696398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:56:25.696398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:25.696398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 22 23:56:25.696398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:25.696398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:25.696398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:25.696398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:25.696398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:25.696398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:25.696398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:25.697460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:25.697460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:25.697460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:25.697460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:25.697460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:25.697460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:25.697460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:25.697460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:56:25.697460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:56:25.697460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:56:25.697460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:56:25.697460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:25.697460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:25.697460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:25.697460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:25.697460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:25.697460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:25.698410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:25.698410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:25.698410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:25.698410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:25.698410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:25.698410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:25.698410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:25.698410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:25.698410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:25.698410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:25.698410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:56:25.698410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:25.698410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 22 23:56:25.698410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:25.702508 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 336/676] 10.222.0.13 () {56 vars in 1132 bytes} [Sat Oct 22 23:56:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:25.703954 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8bb6a-34b3-4249-824f-efa86b4c3099 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:25.789133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4367a0-5f30-43de-ac18-0194ca8c4535 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:25.789633 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4367a0-5f30-43de-ac18-0194ca8c4535 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:25.789633 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4367a0-5f30-43de-ac18-0194ca8c4535 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:25.791184 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 337/677] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:25.798755 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db3c6a-14c3-4a57-bfd7-1a8c907f59ff None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:25.799053 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db3c6a-14c3-4a57-bfd7-1a8c907f59ff None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:25.799343 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db3c6a-14c3-4a57-bfd7-1a8c907f59ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:25.811449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8bb6a-34b3-4249-824f-efa86b4c3099 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:25.824675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-b3db3c6a-14c3-4a57-bfd7-1a8c907f59ff None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:25.826167 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b3db3c6a-14c3-4a57-bfd7-1a8c907f59ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:25.851005 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026cfb4-6113-4845-b8f1-68285903f454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:25.851844 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026cfb4-6113-4845-b8f1-68285903f454 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:25.852347 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026cfb4-6113-4845-b8f1-68285903f454 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:25.852662 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026cfb4-6113-4845-b8f1-68285903f454 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:25.853294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f026cfb4-6113-4845-b8f1-68285903f454 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:25.853601 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f026cfb4-6113-4845-b8f1-68285903f454 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:25.855033 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f026cfb4-6113-4845-b8f1-68285903f454 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:25.860953 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 341/678] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Oct 22 23:56:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:25.866856 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026cfb4-6113-4845-b8f1-68285903f454 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:25.910255 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 338/679] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 22 23:56:25.940627 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:25.989060 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026cfb4-6113-4845-b8f1-68285903f454 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:26.025327 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e997bd8-8e03-4d26-bc40-5467c2ab0ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:26.026270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e997bd8-8e03-4d26-bc40-5467c2ab0ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:26.026768 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e997bd8-8e03-4d26-bc40-5467c2ab0ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:26.027234 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e997bd8-8e03-4d26-bc40-5467c2ab0ca9 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:26.028030 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e997bd8-8e03-4d26-bc40-5467c2ab0ca9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:26.028533 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e997bd8-8e03-4d26-bc40-5467c2ab0ca9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:26.030094 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e997bd8-8e03-4d26-bc40-5467c2ab0ca9 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:26.037218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 342/680] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Oct 22 23:56:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:26.042972 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db3c6a-14c3-4a57-bfd7-1a8c907f59ff None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:26.066429 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:26.076722 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905929d4-af6f-4795-805a-a6f21e6f3836 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:26.077016 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905929d4-af6f-4795-805a-a6f21e6f3836 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:26.077291 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905929d4-af6f-4795-805a-a6f21e6f3836 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:26.078908 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 343/681] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 22 23:56:26.084559 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45024954-aac3-4712-be8c-04470b57b446 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:26.084850 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45024954-aac3-4712-be8c-04470b57b446 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:26.085135 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45024954-aac3-4712-be8c-04470b57b446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:26.104407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-45024954-aac3-4712-be8c-04470b57b446 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:26.105719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-45024954-aac3-4712-be8c-04470b57b446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:26.180214 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db3c6a-14c3-4a57-bfd7-1a8c907f59ff None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:26.192683 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 344/682] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 22 23:56:26.208080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45024954-aac3-4712-be8c-04470b57b446 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:26.229507 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63576b7d-79c4-4eea-a1b6-c42cab096521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:26.230117 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63576b7d-79c4-4eea-a1b6-c42cab096521 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:26.230393 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63576b7d-79c4-4eea-a1b6-c42cab096521 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:26.230662 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63576b7d-79c4-4eea-a1b6-c42cab096521 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:26.231321 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63576b7d-79c4-4eea-a1b6-c42cab096521 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:26.232714 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63576b7d-79c4-4eea-a1b6-c42cab096521 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:26.259703 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 339/683] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Oct 22 23:56:26 2022] POST /identity/v3/endpoints => generated 347 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:56:26.281620 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:26.335856 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45024954-aac3-4712-be8c-04470b57b446 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:26.381671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf727ddb-0bb2-4684-b7df-2b404818e49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:26.382319 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf727ddb-0bb2-4684-b7df-2b404818e49d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:26.382604 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf727ddb-0bb2-4684-b7df-2b404818e49d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:26.382883 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf727ddb-0bb2-4684-b7df-2b404818e49d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:26.394678 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf727ddb-0bb2-4684-b7df-2b404818e49d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:56:26.399736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-bf727ddb-0bb2-4684-b7df-2b404818e49d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 22 23:56:26.399736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:26.399736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:26.399736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:26.399736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:26.399736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:26.399736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:26.399736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:26.399736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:26.399736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:26.399736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:26.399736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:26.399736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:56:26.399736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:56:26.399736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:56:26.399736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:56:26.400761 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:56:26.400761 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:56:26.400761 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:56:26.400761 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:56:26.400761 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:26.400761 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:26.400761 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:26.400761 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:26.400761 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:26.400761 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:26.400761 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:26.400761 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:26.400761 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:26.400761 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:26.400761 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:26.400761 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:26.401626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:26.401626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:26.401626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:26.401626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:26.401626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:56:26.401626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:56:26.401626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:56:26.401626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:56:26.401626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:56:26.401626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:56:26.401626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 22 23:56:26.401626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:26.401626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 345/684] 10.222.0.13 () {56 vars in 1120 bytes} [Sat Oct 22 23:56:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:56:26.407977 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63576b7d-79c4-4eea-a1b6-c42cab096521 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:26.482870 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf727ddb-0bb2-4684-b7df-2b404818e49d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:26.497220 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:26.523288 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63576b7d-79c4-4eea-a1b6-c42cab096521 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:26.558413 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8e5e3-43e2-4511-a151-e55528638bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:26.559185 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8e5e3-43e2-4511-a151-e55528638bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:26.559558 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8e5e3-43e2-4511-a151-e55528638bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:26.559926 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8e5e3-43e2-4511-a151-e55528638bb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:26.560712 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f8e5e3-43e2-4511-a151-e55528638bb9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:26.561108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f8e5e3-43e2-4511-a151-e55528638bb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:26.562554 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f8e5e3-43e2-4511-a151-e55528638bb9 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:26.572088 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 340/685] 10.222.0.13 () {56 vars in 1115 bytes} [Sat Oct 22 23:56:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:26.578052 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8e5e3-43e2-4511-a151-e55528638bb9 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:26.613928 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf727ddb-0bb2-4684-b7df-2b404818e49d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:26.651715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:26.658547 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f330e-54ea-422d-baa9-ec300d267e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:26.659361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f330e-54ea-422d-baa9-ec300d267e94 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:26.659653 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f330e-54ea-422d-baa9-ec300d267e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:26.659938 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f330e-54ea-422d-baa9-ec300d267e94 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:26.660559 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f330e-54ea-422d-baa9-ec300d267e94 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:26.661950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f330e-54ea-422d-baa9-ec300d267e94 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:26.671084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-258f330e-54ea-422d-baa9-ec300d267e94 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 22 23:56:26.671084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:26.671084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:56:26.671084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:56:26.671084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:56:26.671084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:56:26.671084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:56:26.671084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:26.671084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 22 23:56:26.671084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:26.671084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:26.671084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:26.671084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:26.671084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:26.671084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:26.671084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:26.671084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:26.672340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:26.672340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:26.672340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:26.672340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:26.672340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:26.672340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:26.672340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:56:26.672340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:56:26.672340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:56:26.672340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:56:26.672340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:26.672340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:26.672340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:26.672340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:26.672340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:26.672340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:26.673824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:26.673824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:26.673824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:26.673824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:26.673824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:26.673824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:26.673824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:26.673824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:26.673824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:26.673824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:26.673824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:56:26.673824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:26.673824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 22 23:56:26.673824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:26.677913 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 346/686] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Oct 22 23:56:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:26.683483 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f330e-54ea-422d-baa9-ec300d267e94 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:26.700000 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8e5e3-43e2-4511-a151-e55528638bb9 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:26.736374 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59419467-0fbb-4be3-bbe1-0b422ab021f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:26.737150 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59419467-0fbb-4be3-bbe1-0b422ab021f9 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:26.737433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59419467-0fbb-4be3-bbe1-0b422ab021f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:26.737705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59419467-0fbb-4be3-bbe1-0b422ab021f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:26.745510 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:26.750417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59419467-0fbb-4be3-bbe1-0b422ab021f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:26.753275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59419467-0fbb-4be3-bbe1-0b422ab021f9 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:26.769561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-59419467-0fbb-4be3-bbe1-0b422ab021f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:26.769561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:26.769561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:26.769561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:26.769561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:26.769561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:26.769561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:26.769561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:26.769561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:26.769561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:26.769561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:26.769561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:26.769561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:26.769561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:26.769561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:26.769561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:26.770577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:26.770577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:26.770577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:26.770577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:26.770577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:26.770577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:26.770577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:26.770577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:26.770577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:26.770577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:26.770577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:26.770577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:26.770577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:26.770577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:26.770577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:26.770577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:26.770577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:26.771929 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:26.771929 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:26.771929 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:26.771929 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 341/687] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:56:26.776567 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59419467-0fbb-4be3-bbe1-0b422ab021f9 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:26.794498 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f330e-54ea-422d-baa9-ec300d267e94 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:26.836128 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c432383-62eb-433e-a077-529681fb5a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:26.836916 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c432383-62eb-433e-a077-529681fb5a84 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:26.837408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c432383-62eb-433e-a077-529681fb5a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:26.838238 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c432383-62eb-433e-a077-529681fb5a84 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:26.839120 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c432383-62eb-433e-a077-529681fb5a84 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:26.839604 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c432383-62eb-433e-a077-529681fb5a84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:26.841326 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c432383-62eb-433e-a077-529681fb5a84 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:26.848529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 347/688] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Oct 22 23:56:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:26.854229 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c432383-62eb-433e-a077-529681fb5a84 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:26.898246 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59419467-0fbb-4be3-bbe1-0b422ab021f9 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:26.916868 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:26.936341 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be6058-69dd-40a2-97d1-b9bacabc1a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:26.936974 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be6058-69dd-40a2-97d1-b9bacabc1a22 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:26.937356 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be6058-69dd-40a2-97d1-b9bacabc1a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:26.937632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be6058-69dd-40a2-97d1-b9bacabc1a22 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:26.938340 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be6058-69dd-40a2-97d1-b9bacabc1a22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:26.938667 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be6058-69dd-40a2-97d1-b9bacabc1a22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:26.940126 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be6058-69dd-40a2-97d1-b9bacabc1a22 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:26.952761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 342/689] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:26.961507 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be6058-69dd-40a2-97d1-b9bacabc1a22 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:26.991775 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c432383-62eb-433e-a077-529681fb5a84 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:27.018748 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:27.035164 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06198f-5a1c-4118-898d-b360af076fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:27.036386 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06198f-5a1c-4118-898d-b360af076fec None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:27.036934 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06198f-5a1c-4118-898d-b360af076fec None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:27.037426 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06198f-5a1c-4118-898d-b360af076fec None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:27.038383 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e06198f-5a1c-4118-898d-b360af076fec None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:27.038965 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e06198f-5a1c-4118-898d-b360af076fec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:27.040612 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e06198f-5a1c-4118-898d-b360af076fec None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:27.047494 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 348/690] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Oct 22 23:56:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:27.056469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06198f-5a1c-4118-898d-b360af076fec None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:27.084394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be6058-69dd-40a2-97d1-b9bacabc1a22 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:27.130417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b5dd0-9e7d-4b11-ae8c-353684a61159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:27.131059 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b5dd0-9e7d-4b11-ae8c-353684a61159 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:27.131345 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b5dd0-9e7d-4b11-ae8c-353684a61159 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:27.131622 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b5dd0-9e7d-4b11-ae8c-353684a61159 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:27.140817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4b5dd0-9e7d-4b11-ae8c-353684a61159 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:27.143020 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4b5dd0-9e7d-4b11-ae8c-353684a61159 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:27.159562 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-0f4b5dd0-9e7d-4b11-ae8c-353684a61159 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 22 23:56:27.159562 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:27.159562 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:27.159562 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:27.159562 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:27.159562 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:27.159562 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:27.159562 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:27.159562 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:27.159562 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:27.159562 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:27.159562 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:27.159562 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:56:27.159562 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:56:27.159562 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:56:27.159562 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:56:27.160566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:27.160566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:27.160566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:56:27.160566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:27.160566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:56:27.160566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:27.160566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:27.160566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:27.160566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:27.160566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:27.160566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:27.160566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:27.160566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:27.160566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:27.160566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:27.160566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:27.160566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:27.160566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:27.161574 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:27.161574 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:27.161574 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:56:27.161574 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:56:27.161574 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:56:27.161574 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:56:27.161574 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:56:27.161574 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:56:27.161574 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 22 23:56:27.161574 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:27.162124 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06198f-5a1c-4118-898d-b360af076fec None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:27.164575 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 343/691] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Oct 22 23:56:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:27.166271 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b5dd0-9e7d-4b11-ae8c-353684a61159 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:27.168435 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:27.204945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5e998-c4f4-42e6-a3d2-11c043a1bd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:27.205573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5e998-c4f4-42e6-a3d2-11c043a1bd9c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:27.205853 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5e998-c4f4-42e6-a3d2-11c043a1bd9c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:27.206122 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5e998-c4f4-42e6-a3d2-11c043a1bd9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:27.206812 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d5e998-c4f4-42e6-a3d2-11c043a1bd9c None admin] RBAC: Adding query filter params (interface=public, service_id=1b939ddb581e4c528434490d068741dc, region_id=RegionOne) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:27.207120 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d5e998-c4f4-42e6-a3d2-11c043a1bd9c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:27.208466 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d5e998-c4f4-42e6-a3d2-11c043a1bd9c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:27.221771 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 349/692] 10.222.0.13 () {56 vars in 1251 bytes} [Sat Oct 22 23:56:27 2022] GET /identity/v3/endpoints?service_id=1b939ddb581e4c528434490d068741dc&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:27.223246 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5e998-c4f4-42e6-a3d2-11c043a1bd9c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:27.307655 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b5dd0-9e7d-4b11-ae8c-353684a61159 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:27.320336 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:27.331515 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5e998-c4f4-42e6-a3d2-11c043a1bd9c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:27.348735 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36af339-0678-4c65-a3cc-17a55b6ebce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:27.349838 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36af339-0678-4c65-a3cc-17a55b6ebce5 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:27.350326 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36af339-0678-4c65-a3cc-17a55b6ebce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:27.350931 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36af339-0678-4c65-a3cc-17a55b6ebce5 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:27.351866 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36af339-0678-4c65-a3cc-17a55b6ebce5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:27.352505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36af339-0678-4c65-a3cc-17a55b6ebce5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:27.354139 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36af339-0678-4c65-a3cc-17a55b6ebce5 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:27.366527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba576ab5-d2ed-465a-ba1a-97b935d27d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:27.367124 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba576ab5-d2ed-465a-ba1a-97b935d27d1c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:27.367407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba576ab5-d2ed-465a-ba1a-97b935d27d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:27.367680 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba576ab5-d2ed-465a-ba1a-97b935d27d1c None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:27.368275 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba576ab5-d2ed-465a-ba1a-97b935d27d1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:27.369651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba576ab5-d2ed-465a-ba1a-97b935d27d1c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:27.380123 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 350/693] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Oct 22 23:56:27 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 22 23:56:27.380724 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 344/694] 10.222.0.13 () {56 vars in 1149 bytes} [Sat Oct 22 23:56:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:27.381769 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba576ab5-d2ed-465a-ba1a-97b935d27d1c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:27.424004 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:27.508438 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba576ab5-d2ed-465a-ba1a-97b935d27d1c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:27.547032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d794344-93b5-41bf-83af-15ff3584b840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:27.548724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d794344-93b5-41bf-83af-15ff3584b840 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:27.549257 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d794344-93b5-41bf-83af-15ff3584b840 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:27.549748 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d794344-93b5-41bf-83af-15ff3584b840 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:27.605544 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426719b8-09cf-4103-b75e-2b5aebcb4cf4 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:27.605864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426719b8-09cf-4103-b75e-2b5aebcb4cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:27.606110 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426719b8-09cf-4103-b75e-2b5aebcb4cf4 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:27.607738 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 345/695] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 22 23:56:27.613822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a5ff0-3292-4e10-bab5-94e11ce46bc8 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:27.614112 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a5ff0-3292-4e10-bab5-94e11ce46bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:27.614389 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a5ff0-3292-4e10-bab5-94e11ce46bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:27.632247 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d794344-93b5-41bf-83af-15ff3584b840 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:27.634045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d794344-93b5-41bf-83af-15ff3584b840 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:27.635308 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-824a5ff0-3292-4e10-bab5-94e11ce46bc8 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:27.636513 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-824a5ff0-3292-4e10-bab5-94e11ce46bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:27.680591 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:27.744685 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 346/696] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:27.757156 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 351/697] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:56:27 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 22 23:56:27.764545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a5ff0-3292-4e10-bab5-94e11ce46bc8 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:27.863269 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:27.867485 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a5ff0-3292-4e10-bab5-94e11ce46bc8 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:27.904476 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c241e4-7a17-43cd-9926-2afd1d126ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:27.905118 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c241e4-7a17-43cd-9926-2afd1d126ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:27.905402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c241e4-7a17-43cd-9926-2afd1d126ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:27.905681 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c241e4-7a17-43cd-9926-2afd1d126ea3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:27.917406 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c241e4-7a17-43cd-9926-2afd1d126ea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:27.918904 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c241e4-7a17-43cd-9926-2afd1d126ea3 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:27.934174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-63c241e4-7a17-43cd-9926-2afd1d126ea3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:27.934174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:27.934174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:27.934174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:27.934174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:27.934174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:27.934174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:27.934174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:27.934174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:27.934174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:27.934174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:27.934174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:27.934174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:27.934174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:27.934174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:27.934174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:27.935145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:27.935145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:27.935145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:27.935145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:27.935145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:27.935145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:27.935145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:27.935145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:27.935145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:27.935145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:27.935145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:27.935145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:27.935145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:27.935145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:27.935145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:27.935145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:27.935145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:27.936056 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:27.936056 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:27.936056 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:27.936056 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 347/698] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:27.941904 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d794344-93b5-41bf-83af-15ff3584b840 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:28.043391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d794344-93b5-41bf-83af-15ff3584b840 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:28.048302 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c241e4-7a17-43cd-9926-2afd1d126ea3 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:28.075342 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:28.083081 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4558aba-6c93-49f6-983f-7eea4a39f0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:28.083975 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4558aba-6c93-49f6-983f-7eea4a39f0af None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:28.084263 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4558aba-6c93-49f6-983f-7eea4a39f0af None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:28.084721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4558aba-6c93-49f6-983f-7eea4a39f0af None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:28.085687 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4558aba-6c93-49f6-983f-7eea4a39f0af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:28.086065 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4558aba-6c93-49f6-983f-7eea4a39f0af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:28.087827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4558aba-6c93-49f6-983f-7eea4a39f0af None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:28.103639 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 352/699] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:28.110915 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4558aba-6c93-49f6-983f-7eea4a39f0af None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:28.177285 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c241e4-7a17-43cd-9926-2afd1d126ea3 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:28.213025 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16eaf8-2077-4aa7-97f2-4957214f5d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:28.213709 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16eaf8-2077-4aa7-97f2-4957214f5d3b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:28.214082 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16eaf8-2077-4aa7-97f2-4957214f5d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:28.214450 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16eaf8-2077-4aa7-97f2-4957214f5d3b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:28.215144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16eaf8-2077-4aa7-97f2-4957214f5d3b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:28.216770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16eaf8-2077-4aa7-97f2-4957214f5d3b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:28.226743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-1f16eaf8-2077-4aa7-97f2-4957214f5d3b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 22 23:56:28.226743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:28.226743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:56:28.226743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:56:28.226743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:56:28.226743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:56:28.226743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:56:28.226743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:28.226743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 22 23:56:28.226743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:28.226743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:28.226743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:28.226743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:28.226743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:28.226743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:28.226743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:28.226743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:28.228033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:28.228033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:28.228033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:28.228033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:28.228033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:28.228033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:28.228033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:56:28.228033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:56:28.228033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:56:28.228033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:56:28.228033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:28.228033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:28.228033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:28.228033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:28.228033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:28.228033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:28.229169 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:28.229169 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:28.229169 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:28.229169 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:28.229169 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:28.229169 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:28.229169 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:28.229169 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:28.229169 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:28.229169 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:28.229169 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:56:28.229169 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:28.229169 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 22 23:56:28.229169 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:28.236179 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4558aba-6c93-49f6-983f-7eea4a39f0af None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:28.236532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 348/700] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Oct 22 23:56:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:56:28.237980 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16eaf8-2077-4aa7-97f2-4957214f5d3b None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:28.255106 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:28.282817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc8b27-9cb4-47ff-824d-ebcd7e0a8e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:28.284444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc8b27-9cb4-47ff-824d-ebcd7e0a8e14 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:28.284774 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc8b27-9cb4-47ff-824d-ebcd7e0a8e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:28.285050 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc8b27-9cb4-47ff-824d-ebcd7e0a8e14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:28.293098 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc8b27-9cb4-47ff-824d-ebcd7e0a8e14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:28.295133 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc8b27-9cb4-47ff-824d-ebcd7e0a8e14 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:28.310525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-5efc8b27-9cb4-47ff-824d-ebcd7e0a8e14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:28.310525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:28.310525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:28.310525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:28.310525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:28.310525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:28.310525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:28.310525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:28.310525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:28.310525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:28.310525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:28.310525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:28.310525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:56:28.310525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:56:28.310525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:56:28.311689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:56:28.311689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:28.311689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:28.311689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:28.311689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:28.311689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:28.311689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:28.311689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:28.311689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:28.311689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:28.311689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:28.311689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:28.311689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:28.311689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:28.311689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:28.311689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:28.311689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:28.312797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:56:28.312797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:56:28.312797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:56:28.312797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:56:28.312797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:56:28.312797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:56:28.312797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:28.312797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:28.312797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 353/701] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:56:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:28.319156 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc8b27-9cb4-47ff-824d-ebcd7e0a8e14 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:28.372801 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16eaf8-2077-4aa7-97f2-4957214f5d3b None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:28.382735 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:28.419741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d44da3-d3f1-4250-b4c2-cd41a4a5a06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:28.420957 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d44da3-d3f1-4250-b4c2-cd41a4a5a06a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:28.421876 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d44da3-d3f1-4250-b4c2-cd41a4a5a06a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:28.422366 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d44da3-d3f1-4250-b4c2-cd41a4a5a06a None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:28.423810 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d44da3-d3f1-4250-b4c2-cd41a4a5a06a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:28.424354 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d44da3-d3f1-4250-b4c2-cd41a4a5a06a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:28.427079 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d44da3-d3f1-4250-b4c2-cd41a4a5a06a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:28.440392 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 349/702] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Oct 22 23:56:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:28.442721 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d44da3-d3f1-4250-b4c2-cd41a4a5a06a None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:28.445432 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc8b27-9cb4-47ff-824d-ebcd7e0a8e14 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:28.486453 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c90230-0db1-4a47-b725-8825a2a74598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:28.486960 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c90230-0db1-4a47-b725-8825a2a74598 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:28.487244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c90230-0db1-4a47-b725-8825a2a74598 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:28.487517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c90230-0db1-4a47-b725-8825a2a74598 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:28.488205 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c90230-0db1-4a47-b725-8825a2a74598 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:28.488537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c90230-0db1-4a47-b725-8825a2a74598 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:28.489966 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c90230-0db1-4a47-b725-8825a2a74598 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:28.508633 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 354/703] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Oct 22 23:56:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:28.509545 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c90230-0db1-4a47-b725-8825a2a74598 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:28.533367 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:28.557893 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d44da3-d3f1-4250-b4c2-cd41a4a5a06a None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:28.593927 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10832c-c381-48db-aff9-507125f80ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:28.594539 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10832c-c381-48db-aff9-507125f80ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:28.595781 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10832c-c381-48db-aff9-507125f80ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:28.596064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10832c-c381-48db-aff9-507125f80ea9 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:28.597013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e10832c-c381-48db-aff9-507125f80ea9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:28.597331 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e10832c-c381-48db-aff9-507125f80ea9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:28.598851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e10832c-c381-48db-aff9-507125f80ea9 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:28.608258 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 350/704] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Oct 22 23:56:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:28.608258 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640b8e7-621e-46fd-9719-1fa757d1882c None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:28.608258 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640b8e7-621e-46fd-9719-1fa757d1882c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:28.608258 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640b8e7-621e-46fd-9719-1fa757d1882c None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:28.608258 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 351/705] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:28.611899 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640b8e7-621e-46fd-9719-1fa757d1882c None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:28.625989 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c90230-0db1-4a47-b725-8825a2a74598 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:28.640688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:28.667233 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a64c5d-5272-43b4-b4f4-a2bd9a66649e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:28.667842 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a64c5d-5272-43b4-b4f4-a2bd9a66649e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:28.668121 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a64c5d-5272-43b4-b4f4-a2bd9a66649e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:28.668394 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a64c5d-5272-43b4-b4f4-a2bd9a66649e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:28.669140 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a64c5d-5272-43b4-b4f4-a2bd9a66649e None admin] RBAC: Adding query filter params (role.id=348c1c2575e44dc7a916de12f69abca6, scope.project.id=5c9246c059754504afee4711d113a3fe, user.id=7c0356072f92451f98b03a2225e22f86) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:28.669451 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a64c5d-5272-43b4-b4f4-a2bd9a66649e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:28.670860 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a64c5d-5272-43b4-b4f4-a2bd9a66649e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:28.677785 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 355/706] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:56:28 2022] GET /identity/v3/role_assignments?user.id=7c0356072f92451f98b03a2225e22f86&scope.project.id=5c9246c059754504afee4711d113a3fe&role.id=348c1c2575e44dc7a916de12f69abca6 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:28.679035 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e83351-5ce2-4a8a-a78c-638b3c3af2a9 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:28.679323 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e83351-5ce2-4a8a-a78c-638b3c3af2a9 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:28.679596 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e83351-5ce2-4a8a-a78c-638b3c3af2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:28.703599 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-e0e83351-5ce2-4a8a-a78c-638b3c3af2a9 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:28.705121 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e0e83351-5ce2-4a8a-a78c-638b3c3af2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:28.735367 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640b8e7-621e-46fd-9719-1fa757d1882c None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:28.783066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeedaa8-e6c7-4199-b79d-db0cac69e59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:28.784781 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeedaa8-e6c7-4199-b79d-db0cac69e59c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:28.785272 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeedaa8-e6c7-4199-b79d-db0cac69e59c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:28.785696 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeedaa8-e6c7-4199-b79d-db0cac69e59c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:28.786776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeedaa8-e6c7-4199-b79d-db0cac69e59c None admin] RBAC: Adding query filter params (interface=public, service_id=3a70e1a833bd44e0aa0b813378833846, region_id=RegionOne) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:28.787357 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeedaa8-e6c7-4199-b79d-db0cac69e59c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:28.788909 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 356/707] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:28.789267 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeedaa8-e6c7-4199-b79d-db0cac69e59c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:28.800187 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 352/708] 10.222.0.13 () {56 vars in 1251 bytes} [Sat Oct 22 23:56:28 2022] GET /identity/v3/endpoints?service_id=3a70e1a833bd44e0aa0b813378833846&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:28.801607 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e83351-5ce2-4a8a-a78c-638b3c3af2a9 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:28.917889 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e83351-5ce2-4a8a-a78c-638b3c3af2a9 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:28.926408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:28.957579 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0796498-2a95-4f45-9e73-567afff98956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:28.958179 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0796498-2a95-4f45-9e73-567afff98956 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:28.958468 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0796498-2a95-4f45-9e73-567afff98956 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:28.958746 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0796498-2a95-4f45-9e73-567afff98956 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:28.959353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0796498-2a95-4f45-9e73-567afff98956 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:28.960814 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0796498-2a95-4f45-9e73-567afff98956 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:28.967250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 357/709] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Oct 22 23:56:28 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 22 23:56:29.060222 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeedaa8-e6c7-4199-b79d-db0cac69e59c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:29.070725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:29.175384 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeedaa8-e6c7-4199-b79d-db0cac69e59c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:29.211515 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181502b-efef-487b-9424-88801e310686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:29.212149 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181502b-efef-487b-9424-88801e310686 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:29.212428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181502b-efef-487b-9424-88801e310686 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:29.212735 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181502b-efef-487b-9424-88801e310686 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:29.213322 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181502b-efef-487b-9424-88801e310686 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:29.214680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181502b-efef-487b-9424-88801e310686 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:29.224743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-6181502b-efef-487b-9424-88801e310686 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 22 23:56:29.224743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:29.224743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:56:29.224743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:56:29.224743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:56:29.224743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:56:29.224743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:56:29.224743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:29.224743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 22 23:56:29.224743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:29.224743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:29.224743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:29.224743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:29.224743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:29.224743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:29.224743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:29.224743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:29.225817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:29.225817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:29.225817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:29.225817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:29.225817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:29.225817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:29.225817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:56:29.225817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:56:29.225817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:56:29.225817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:56:29.225817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:29.225817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:29.225817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:29.225817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:29.225817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:29.225817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:29.227189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:29.227189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:29.227189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:29.227189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:29.227189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:29.227189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:29.227189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:29.227189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:29.227189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:29.227189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:29.227189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:56:29.227189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:29.227189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 22 23:56:29.227189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:29.227189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 353/710] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:56:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:29.231982 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0796498-2a95-4f45-9e73-567afff98956 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:29.338202 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0796498-2a95-4f45-9e73-567afff98956 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:29.374008 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf8b94-7fe4-4091-8f32-ea2913240c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:29.374622 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf8b94-7fe4-4091-8f32-ea2913240c9d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:29.374905 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf8b94-7fe4-4091-8f32-ea2913240c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:29.375181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf8b94-7fe4-4091-8f32-ea2913240c9d None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:29.375804 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf8b94-7fe4-4091-8f32-ea2913240c9d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:29.376108 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf8b94-7fe4-4091-8f32-ea2913240c9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:29.378257 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf8b94-7fe4-4091-8f32-ea2913240c9d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:29.383676 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 358/711] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Oct 22 23:56:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:29.389290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181502b-efef-487b-9424-88801e310686 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:29.463560 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:29.498376 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181502b-efef-487b-9424-88801e310686 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:29.534424 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26694798-c873-4b5a-8ef0-2216a8ed6fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:29.535033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26694798-c873-4b5a-8ef0-2216a8ed6fce None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:29.535313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26694798-c873-4b5a-8ef0-2216a8ed6fce None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:29.535587 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26694798-c873-4b5a-8ef0-2216a8ed6fce None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:29.536215 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26694798-c873-4b5a-8ef0-2216a8ed6fce None admin] RBAC: Adding query filter params (type=compute) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:29.536541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26694798-c873-4b5a-8ef0-2216a8ed6fce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:29.537906 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26694798-c873-4b5a-8ef0-2216a8ed6fce None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:29.542938 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 354/712] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Oct 22 23:56:29 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:29.771248 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433688d-c445-4081-ac78-5004b2180d16 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:29.771456 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433688d-c445-4081-ac78-5004b2180d16 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:29.771737 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433688d-c445-4081-ac78-5004b2180d16 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:29.773441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 359/713] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:29.778796 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417e5a6-096b-4ccd-9c58-6e4875869ff2 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:29.779093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417e5a6-096b-4ccd-9c58-6e4875869ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:29.779366 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417e5a6-096b-4ccd-9c58-6e4875869ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:29.797351 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-6417e5a6-096b-4ccd-9c58-6e4875869ff2 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:29.799048 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-6417e5a6-096b-4ccd-9c58-6e4875869ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:29.867962 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 355/714] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:30.137811 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433688d-c445-4081-ac78-5004b2180d16 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:30.175341 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:30.247938 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433688d-c445-4081-ac78-5004b2180d16 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:30.289384 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bae82-a1c6-4b9f-bba6-f8114e5c789f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:30.290030 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bae82-a1c6-4b9f-bba6-f8114e5c789f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:30.290312 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bae82-a1c6-4b9f-bba6-f8114e5c789f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:30.290584 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bae82-a1c6-4b9f-bba6-f8114e5c789f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:30.291165 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268bae82-a1c6-4b9f-bba6-f8114e5c789f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:30.292546 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268bae82-a1c6-4b9f-bba6-f8114e5c789f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:30.301906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-268bae82-a1c6-4b9f-bba6-f8114e5c789f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 22 23:56:30.301906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:30.301906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:56:30.301906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:56:30.301906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:56:30.301906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:56:30.301906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:56:30.301906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:30.301906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 22 23:56:30.301906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:30.301906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:30.301906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:30.301906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:30.301906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:30.301906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:30.301906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:30.301906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:30.302932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:30.302932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:30.302932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:30.302932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:30.302932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:30.302932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:30.302932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:56:30.302932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:56:30.302932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:56:30.302932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:56:30.302932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:30.302932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:30.302932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:30.302932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:30.302932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:30.302932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:30.303793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:30.303793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:30.303793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:30.303793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:30.303793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:30.303793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:30.303793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:30.303793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:30.303793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:30.303793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:30.303793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:56:30.303793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:30.303793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 22 23:56:30.303793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:30.303793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 360/715] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Oct 22 23:56:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:30.309218 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417e5a6-096b-4ccd-9c58-6e4875869ff2 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:30.375544 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:30.414506 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417e5a6-096b-4ccd-9c58-6e4875869ff2 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:30.448788 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5774688-5fe7-4e21-83f7-12fa1acdca2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:30.449463 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5774688-5fe7-4e21-83f7-12fa1acdca2b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:30.449746 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5774688-5fe7-4e21-83f7-12fa1acdca2b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:30.450022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5774688-5fe7-4e21-83f7-12fa1acdca2b None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:30.450873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5774688-5fe7-4e21-83f7-12fa1acdca2b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:30.451227 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5774688-5fe7-4e21-83f7-12fa1acdca2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:30.452621 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5774688-5fe7-4e21-83f7-12fa1acdca2b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:30.458502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 356/716] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Oct 22 23:56:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:30.464435 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bae82-a1c6-4b9f-bba6-f8114e5c789f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:30.568780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:30.572970 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bae82-a1c6-4b9f-bba6-f8114e5c789f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:30.608255 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a4a158-f6ff-4b54-a356-72c1eab8f1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:30.608893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4a158-f6ff-4b54-a356-72c1eab8f1ee None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:30.609176 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4a158-f6ff-4b54-a356-72c1eab8f1ee None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:30.609471 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4a158-f6ff-4b54-a356-72c1eab8f1ee None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:30.610138 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a4a158-f6ff-4b54-a356-72c1eab8f1ee None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:30.610443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a4a158-f6ff-4b54-a356-72c1eab8f1ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:30.612497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a4a158-f6ff-4b54-a356-72c1eab8f1ee None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:30.617937 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 361/717] 10.222.0.13 () {56 vars in 1114 bytes} [Sat Oct 22 23:56:30 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:30.623664 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5774688-5fe7-4e21-83f7-12fa1acdca2b None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:30.728123 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5774688-5fe7-4e21-83f7-12fa1acdca2b None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:30.756396 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:30.764776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a0488-c680-47b3-8e5a-9b33d7c93c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:30.765471 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a0488-c680-47b3-8e5a-9b33d7c93c99 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:30.766223 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a0488-c680-47b3-8e5a-9b33d7c93c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:30.766894 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a0488-c680-47b3-8e5a-9b33d7c93c99 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:30.768131 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a0488-c680-47b3-8e5a-9b33d7c93c99 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:30.769544 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a0488-c680-47b3-8e5a-9b33d7c93c99 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:30.799730 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 357/718] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Oct 22 23:56:30 2022] POST /identity/v3/endpoints => generated 346 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:56:31.118857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55400a4a-8a08-40fb-98f8-6b64b528d0ea None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:31.119144 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55400a4a-8a08-40fb-98f8-6b64b528d0ea None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:31.119421 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55400a4a-8a08-40fb-98f8-6b64b528d0ea None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:31.121064 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 362/719] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:31.126569 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a1103-66a3-4c0f-9937-c895a389c6b4 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:31.126926 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a1103-66a3-4c0f-9937-c895a389c6b4 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:31.127136 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a1103-66a3-4c0f-9937-c895a389c6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:31.145763 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-1c3a1103-66a3-4c0f-9937-c895a389c6b4 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:31.146940 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-1c3a1103-66a3-4c0f-9937-c895a389c6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:31.232589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 358/720] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:31.312233 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368fed1-e57b-486e-b5c3-a325e2666d5a None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:31.313225 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368fed1-e57b-486e-b5c3-a325e2666d5a None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:31.313793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368fed1-e57b-486e-b5c3-a325e2666d5a None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:31.316130 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 363/721] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:31.322007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774c3fa-33b4-476e-b1b8-de161d877367 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:31.322861 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774c3fa-33b4-476e-b1b8-de161d877367 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:31.323341 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774c3fa-33b4-476e-b1b8-de161d877367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:31.342084 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-b774c3fa-33b4-476e-b1b8-de161d877367 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:31.343626 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b774c3fa-33b4-476e-b1b8-de161d877367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:31.418739 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 359/722] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:31.513224 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368fed1-e57b-486e-b5c3-a325e2666d5a None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:31.620964 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368fed1-e57b-486e-b5c3-a325e2666d5a None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:31.658084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d2f4c-fd5f-4c1c-99ec-a4e796137550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:31.658721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d2f4c-fd5f-4c1c-99ec-a4e796137550 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:31.659008 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d2f4c-fd5f-4c1c-99ec-a4e796137550 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:31.659284 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d2f4c-fd5f-4c1c-99ec-a4e796137550 None admin] PATH_INFO: `/v3/services/image` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:31.659885 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d2f4c-fd5f-4c1c-99ec-a4e796137550 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:31.661278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d2f4c-fd5f-4c1c-99ec-a4e796137550 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:31.671198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-d87d2f4c-fd5f-4c1c-99ec-a4e796137550 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 22 23:56:31.671198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:31.671198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:56:31.671198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:56:31.671198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:56:31.671198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:56:31.671198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:56:31.671198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:31.671198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 22 23:56:31.671198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:31.671198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:31.671198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:31.671198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:31.671198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:31.671198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:31.671198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:31.671198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:31.672250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:31.672250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:31.672250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:31.672250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:31.672250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:31.672250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:31.672250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:56:31.672250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:56:31.672250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:56:31.672250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:56:31.672250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:31.672250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:31.672250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:31.672250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:31.672250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:31.672250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:31.673210 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:31.673210 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:31.673210 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:31.673210 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:31.673210 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:31.673210 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:31.673210 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:31.673210 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:31.673210 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:31.673210 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:31.673210 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:56:31.673210 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:31.673210 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 22 23:56:31.673210 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:31.673210 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 364/723] 10.222.0.13 () {56 vars in 1105 bytes} [Sat Oct 22 23:56:31 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:31.682077 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774c3fa-33b4-476e-b1b8-de161d877367 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:31.699221 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d2f4c-fd5f-4c1c-99ec-a4e796137550 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:31.731115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:31.744832 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:31.797887 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774c3fa-33b4-476e-b1b8-de161d877367 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:31.833818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d2f4c-fd5f-4c1c-99ec-a4e796137550 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:31.838463 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664a8a7-8c49-4828-a884-fc733f111349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:31.839067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664a8a7-8c49-4828-a884-fc733f111349 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:31.839344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664a8a7-8c49-4828-a884-fc733f111349 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:31.839615 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664a8a7-8c49-4828-a884-fc733f111349 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:31.840236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2664a8a7-8c49-4828-a884-fc733f111349 None admin] RBAC: Adding query filter params (name=image) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:31.840556 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2664a8a7-8c49-4828-a884-fc733f111349 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:31.841937 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2664a8a7-8c49-4828-a884-fc733f111349 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:31.846753 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 360/724] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:31.850021 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9904a-4b2c-49f0-9ed6-253cccec062e None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:31.851314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9904a-4b2c-49f0-9ed6-253cccec062e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:31.851608 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9904a-4b2c-49f0-9ed6-253cccec062e None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:31.853298 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 361/725] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:31.853784 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9904a-4b2c-49f0-9ed6-253cccec062e None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:31.875719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6075bfa-8eb7-4bff-b29d-4f7409d0f3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:31.876938 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6075bfa-8eb7-4bff-b29d-4f7409d0f3ca None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:31.877225 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6075bfa-8eb7-4bff-b29d-4f7409d0f3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:31.877490 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6075bfa-8eb7-4bff-b29d-4f7409d0f3ca None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:31.878067 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6075bfa-8eb7-4bff-b29d-4f7409d0f3ca None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:31.879427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6075bfa-8eb7-4bff-b29d-4f7409d0f3ca None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:31.888412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-b6075bfa-8eb7-4bff-b29d-4f7409d0f3ca None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 22 23:56:31.888412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:31.888412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:56:31.888412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:56:31.888412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:56:31.888412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:56:31.888412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:56:31.888412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:31.888412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 22 23:56:31.888412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:31.888412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:31.888412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:31.888412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:31.888412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:31.888412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:31.888412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:31.888412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:31.889495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:31.889495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:31.889495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:31.889495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:31.889495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:31.889495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:31.889495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:56:31.889495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:56:31.889495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:56:31.889495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:56:31.889495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:31.889495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:31.889495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:31.889495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:31.889495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:31.889495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:31.890998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:31.890998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:31.890998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:31.890998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:31.890998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:31.890998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:31.890998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:31.890998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:31.890998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:31.890998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:31.890998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:56:31.890998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:31.890998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 22 23:56:31.890998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:31.890998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 365/726] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Oct 22 23:56:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:56:31.891890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6655cf3-42c9-4db5-832c-2f7b25509fb3 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:31.891890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6655cf3-42c9-4db5-832c-2f7b25509fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:31.891890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6655cf3-42c9-4db5-832c-2f7b25509fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:31.910300 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-b6655cf3-42c9-4db5-832c-2f7b25509fb3 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:31.911497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b6655cf3-42c9-4db5-832c-2f7b25509fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:31.965325 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9904a-4b2c-49f0-9ed6-253cccec062e None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:31.968628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:31.983255 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 366/727] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:31.984155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6655cf3-42c9-4db5-832c-2f7b25509fb3 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:32.010575 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:32.021873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5e293-fa6f-48b6-a9f3-5e4426747c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:32.023061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5e293-fa6f-48b6-a9f3-5e4426747c6c None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:32.023553 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5e293-fa6f-48b6-a9f3-5e4426747c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:32.024030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5e293-fa6f-48b6-a9f3-5e4426747c6c None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:32.025090 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad5e293-fa6f-48b6-a9f3-5e4426747c6c None admin] RBAC: Adding query filter params (type=image) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:32.025407 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad5e293-fa6f-48b6-a9f3-5e4426747c6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:32.026764 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad5e293-fa6f-48b6-a9f3-5e4426747c6c None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:32.035885 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 362/728] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:31 2022] GET /identity/v3/services?type=image => generated 122 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:32.109654 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6655cf3-42c9-4db5-832c-2f7b25509fb3 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:32.151695 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215c644-a4e7-4760-9aab-98ab9a433f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:32.152330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215c644-a4e7-4760-9aab-98ab9a433f92 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:32.152648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215c644-a4e7-4760-9aab-98ab9a433f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:32.152932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215c644-a4e7-4760-9aab-98ab9a433f92 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:32.153583 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d215c644-a4e7-4760-9aab-98ab9a433f92 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:32.153895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d215c644-a4e7-4760-9aab-98ab9a433f92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:32.155282 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d215c644-a4e7-4760-9aab-98ab9a433f92 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:32.161565 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 367/729] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Oct 22 23:56:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:32.174719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5e293-fa6f-48b6-a9f3-5e4426747c6c None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:32.283899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215c644-a4e7-4760-9aab-98ab9a433f92 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:32.293804 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5e293-fa6f-48b6-a9f3-5e4426747c6c None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:32.332176 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fedb12-73d7-46f9-a7e1-0d4f823590c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:32.333694 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fedb12-73d7-46f9-a7e1-0d4f823590c0 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:32.334192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fedb12-73d7-46f9-a7e1-0d4f823590c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:32.334670 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fedb12-73d7-46f9-a7e1-0d4f823590c0 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:32.335503 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fedb12-73d7-46f9-a7e1-0d4f823590c0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:32.336095 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fedb12-73d7-46f9-a7e1-0d4f823590c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:32.337839 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fedb12-73d7-46f9-a7e1-0d4f823590c0 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:32.343862 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 363/730] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Oct 22 23:56:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:32.351130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fedb12-73d7-46f9-a7e1-0d4f823590c0 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:32.373328 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:32.390167 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:32.390338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215c644-a4e7-4760-9aab-98ab9a433f92 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:32.429803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a6077-83b0-4d7c-b4bb-ef9f3a824648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:32.430551 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a6077-83b0-4d7c-b4bb-ef9f3a824648 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:32.430809 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a6077-83b0-4d7c-b4bb-ef9f3a824648 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:32.431090 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a6077-83b0-4d7c-b4bb-ef9f3a824648 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:32.431749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a6077-83b0-4d7c-b4bb-ef9f3a824648 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:32.433247 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a6077-83b0-4d7c-b4bb-ef9f3a824648 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:32.457015 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 368/731] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Oct 22 23:56:32 2022] POST /identity/v3/services => generated 242 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 22 23:56:32.474887 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fedb12-73d7-46f9-a7e1-0d4f823590c0 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:32.523600 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63971c-0dfa-45bb-85d2-48e0e4d889c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:32.524700 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63971c-0dfa-45bb-85d2-48e0e4d889c0 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:32.525204 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63971c-0dfa-45bb-85d2-48e0e4d889c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:32.525680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63971c-0dfa-45bb-85d2-48e0e4d889c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:32.526597 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e63971c-0dfa-45bb-85d2-48e0e4d889c0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:32.528196 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e63971c-0dfa-45bb-85d2-48e0e4d889c0 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:32.551963 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 364/732] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Oct 22 23:56:32 2022] POST /identity/v3/endpoints => generated 349 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 22 23:56:33.099551 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c8e17-6ec3-491d-947f-9bcd0758671e None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:33.099956 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c8e17-6ec3-491d-947f-9bcd0758671e None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:33.100135 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c8e17-6ec3-491d-947f-9bcd0758671e None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:33.114194 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 369/733] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:33 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:33.116245 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6e759-c658-4671-84a7-752a1bf4e3f5 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:33.116437 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6e759-c658-4671-84a7-752a1bf4e3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:33.116730 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6e759-c658-4671-84a7-752a1bf4e3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:33.136683 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-5dd6e759-c658-4671-84a7-752a1bf4e3f5 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:33.138218 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-5dd6e759-c658-4671-84a7-752a1bf4e3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:33.212707 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 365/734] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:33 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:33.484042 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c8e17-6ec3-491d-947f-9bcd0758671e None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:33.594504 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c8e17-6ec3-491d-947f-9bcd0758671e None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:33.629776 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c16af-a123-4372-a4ea-0e9c3550e42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:33.630446 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c16af-a123-4372-a4ea-0e9c3550e42d None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:33.630739 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c16af-a123-4372-a4ea-0e9c3550e42d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:33.631019 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c16af-a123-4372-a4ea-0e9c3550e42d None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:33.631678 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c16af-a123-4372-a4ea-0e9c3550e42d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:33.633124 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c16af-a123-4372-a4ea-0e9c3550e42d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:33.635726 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: WARNING keystone.server.flask.common [None req-a34c16af-a123-4372-a4ea-0e9c3550e42d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 22 23:56:33.650499 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:33.654524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.sql.core [None req-a34c16af-a123-4372-a4ea-0e9c3550e42d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 22 23:56:33.654524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 22 23:56:33.654524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [parameters: {'id': 'fbe2bebf302a4cc4b3b9043bc8495adb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 22 23:56:33.654524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81381) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 22 23:56:33.663779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-a34c16af-a123-4372-a4ea-0e9c3550e42d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fbe2bebf302a4cc4b3b9043bc8495adb.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fbe2bebf302a4cc4b3b9043bc8495adb. Oct 22 23:56:33.663779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:33.663779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 22 23:56:33.663779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 22 23:56:33.663779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 22 23:56:33.663779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 22 23:56:33.663779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 22 23:56:33.663779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result = self._query(query) Oct 22 23:56:33.663779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 22 23:56:33.663779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application conn.query(q) Oct 22 23:56:33.663779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 22 23:56:33.663779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 22 23:56:33.663779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 22 23:56:33.664801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result.read() Oct 22 23:56:33.664801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 22 23:56:33.664801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 22 23:56:33.664801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 22 23:56:33.664801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 22 23:56:33.664801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 22 23:56:33.664801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 22 23:56:33.664801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 22 23:56:33.664801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 22 23:56:33.664801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 22 23:56:33.664801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:33.664801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 22 23:56:33.664801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:33.664801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:33.664801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 22 23:56:33.664801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 22 23:56:33.664801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 22 23:56:33.664801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 22 23:56:33.666017 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 22 23:56:33.666017 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application next(self.gen) Oct 22 23:56:33.666017 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 22 23:56:33.666017 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application yield resource Oct 22 23:56:33.666017 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 22 23:56:33.666017 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application next(self.gen) Oct 22 23:56:33.666017 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 22 23:56:33.666017 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.session.rollback() Oct 22 23:56:33.666017 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 22 23:56:33.666017 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.force_reraise() Oct 22 23:56:33.666017 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 22 23:56:33.666017 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise self.value Oct 22 23:56:33.666017 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 22 23:56:33.666017 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 22 23:56:33.666017 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 22 23:56:33.666017 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application session.commit() Oct 22 23:56:33.666017 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 22 23:56:33.667157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 22 23:56:33.667157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 22 23:56:33.667157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._prepare_impl() Oct 22 23:56:33.667157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 22 23:56:33.667157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.session.flush() Oct 22 23:56:33.667157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 22 23:56:33.667157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._flush(objects) Oct 22 23:56:33.667157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 22 23:56:33.667157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 22 23:56:33.667157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 22 23:56:33.667157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application compat.raise_( Oct 22 23:56:33.667157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 22 23:56:33.667157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception Oct 22 23:56:33.667157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 22 23:56:33.667157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application flush_context.execute() Oct 22 23:56:33.667157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 22 23:56:33.667157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rec.execute(self) Oct 22 23:56:33.668294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 22 23:56:33.668294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 22 23:56:33.668294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 22 23:56:33.668294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 22 23:56:33.668294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 22 23:56:33.668294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 22 23:56:33.668294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 22 23:56:33.668294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 22 23:56:33.668294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 22 23:56:33.668294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 22 23:56:33.668294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 22 23:56:33.668294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 22 23:56:33.668294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 22 23:56:33.668294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 22 23:56:33.668294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 22 23:56:33.670270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 22 23:56:33.670270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 22 23:56:33.670270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception Oct 22 23:56:33.670270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 22 23:56:33.670270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 22 23:56:33.670270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 22 23:56:33.670270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 22 23:56:33.670270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 22 23:56:33.670270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result = self._query(query) Oct 22 23:56:33.670270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 22 23:56:33.670270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application conn.query(q) Oct 22 23:56:33.670270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 22 23:56:33.670270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 22 23:56:33.670270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 22 23:56:33.670270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result.read() Oct 22 23:56:33.670270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 22 23:56:33.670270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 22 23:56:33.672293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 22 23:56:33.672293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 22 23:56:33.672293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 22 23:56:33.672293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 22 23:56:33.672293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 22 23:56:33.672293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 22 23:56:33.672293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 22 23:56:33.672293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 22 23:56:33.672293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [parameters: {'id': 'fbe2bebf302a4cc4b3b9043bc8495adb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 22 23:56:33.672293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 22 23:56:33.672293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:33.672293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:33.672293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:33.672293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:33.672293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 22 23:56:33.674922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 22 23:56:33.674922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 22 23:56:33.674922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 22 23:56:33.674922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 22 23:56:33.674922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:33.674922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:33.674922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:33.674922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:33.674922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:33.674922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:33.674922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:33.674922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:33.674922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:33.674922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:33.674922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:33.674922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:33.674922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:33.677493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:33.677493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 22 23:56:33.677493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 22 23:56:33.677493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:33.677493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:33.677493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 22 23:56:33.677493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 22 23:56:33.677493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fbe2bebf302a4cc4b3b9043bc8495adb. Oct 22 23:56:33.677493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:33.682487 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 370/735] 10.222.0.13 () {60 vars in 1140 bytes} [Sat Oct 22 23:56:33 2022] POST /identity/v3/projects => generated 262 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:33.687628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd6e759-c658-4671-84a7-752a1bf4e3f5 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:33.699765 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:33.793891 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd6e759-c658-4671-84a7-752a1bf4e3f5 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:33.832274 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf96b87-b1b8-4e18-8b49-1fb0fa7bf07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:33.832944 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf96b87-b1b8-4e18-8b49-1fb0fa7bf07a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:33.833230 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf96b87-b1b8-4e18-8b49-1fb0fa7bf07a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:33.833511 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf96b87-b1b8-4e18-8b49-1fb0fa7bf07a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:33.840351 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf96b87-b1b8-4e18-8b49-1fb0fa7bf07a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:33.841893 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf96b87-b1b8-4e18-8b49-1fb0fa7bf07a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:33.856099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-fdf96b87-b1b8-4e18-8b49-1fb0fa7bf07a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:33.856099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:33.856099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:33.856099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:33.856099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:33.856099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:33.856099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:33.856099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:33.856099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:33.856099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:33.856099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:33.856099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:33.856099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:56:33.856099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:56:33.856099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:56:33.857062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:56:33.857062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:33.857062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:33.857062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:33.857062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:33.857062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:33.857062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:33.857062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:33.857062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:33.857062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:33.857062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:33.857062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:33.857062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:33.857062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:33.857062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:33.857062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:33.857062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:33.857958 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:56:33.857958 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:56:33.857958 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:56:33.857958 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:56:33.857958 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:56:33.857958 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:56:33.857958 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:33.857958 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:33.857958 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 366/736] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:56:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:33.865123 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c16af-a123-4372-a4ea-0e9c3550e42d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:33.971296 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c16af-a123-4372-a4ea-0e9c3550e42d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:34.007130 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c6b49-ed35-4c11-bbb8-b22be3a83903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:34.008090 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c6b49-ed35-4c11-bbb8-b22be3a83903 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:34.008598 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c6b49-ed35-4c11-bbb8-b22be3a83903 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:34.009076 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c6b49-ed35-4c11-bbb8-b22be3a83903 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:34.010010 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c6b49-ed35-4c11-bbb8-b22be3a83903 None admin] RBAC: Adding query filter params (name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:34.010531 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c6b49-ed35-4c11-bbb8-b22be3a83903 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:34.012391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c6b49-ed35-4c11-bbb8-b22be3a83903 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:34.032371 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 371/737] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Oct 22 23:56:33 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:34.267652 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129ff5c-5097-49a2-866b-f1420ce478ce None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:34.268000 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129ff5c-5097-49a2-866b-f1420ce478ce None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:34.268329 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129ff5c-5097-49a2-866b-f1420ce478ce None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:34.270046 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 367/738] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:34.275977 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fa310-72c9-445e-9e59-4078b9040f36 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:34.276269 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fa310-72c9-445e-9e59-4078b9040f36 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:34.276611 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fa310-72c9-445e-9e59-4078b9040f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:34.297038 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-db4fa310-72c9-445e-9e59-4078b9040f36 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:34.299174 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-db4fa310-72c9-445e-9e59-4078b9040f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:34.364330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:34.376506 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 372/739] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:34 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:34.644155 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129ff5c-5097-49a2-866b-f1420ce478ce None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:34.720725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:34.740787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052422fb-7b2b-4574-b142-c937e9e03ca4 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:34.741285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052422fb-7b2b-4574-b142-c937e9e03ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:34.741366 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052422fb-7b2b-4574-b142-c937e9e03ca4 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:34.743185 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 373/740] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 22 23:56:34.748095 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd61b7-48d0-4927-a850-631253ee45a1 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:34.748389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd61b7-48d0-4927-a850-631253ee45a1 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:34.748768 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd61b7-48d0-4927-a850-631253ee45a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:34.757939 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129ff5c-5097-49a2-866b-f1420ce478ce None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:34.768059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-bedd61b7-48d0-4927-a850-631253ee45a1 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:34.769211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-bedd61b7-48d0-4927-a850-631253ee45a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:34.797375 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43444344-5c7b-4c42-8bb6-a67df68d186d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:34.799011 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43444344-5c7b-4c42-8bb6-a67df68d186d None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:34.799450 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43444344-5c7b-4c42-8bb6-a67df68d186d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:34.799998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43444344-5c7b-4c42-8bb6-a67df68d186d None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:34.800920 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43444344-5c7b-4c42-8bb6-a67df68d186d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:34.802599 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43444344-5c7b-4c42-8bb6-a67df68d186d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:34.821324 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 368/741] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Oct 22 23:56:34 2022] POST /identity/v3/services => generated 242 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:56:34.873031 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 374/742] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:34 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 22 23:56:34.907095 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51069eac-087f-4360-84a8-5b53a331a81a None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:34.907301 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51069eac-087f-4360-84a8-5b53a331a81a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:34.907979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51069eac-087f-4360-84a8-5b53a331a81a None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:34.909713 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 369/743] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:34.915414 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57231e-b81f-4ac4-b37d-853614497380 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:34.915714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57231e-b81f-4ac4-b37d-853614497380 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:34.915994 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f57231e-b81f-4ac4-b37d-853614497380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:34.936395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-8f57231e-b81f-4ac4-b37d-853614497380 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:34.937939 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-8f57231e-b81f-4ac4-b37d-853614497380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:35.018778 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 375/744] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:34 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:35.152216 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51069eac-087f-4360-84a8-5b53a331a81a None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:35.260894 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51069eac-087f-4360-84a8-5b53a331a81a None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:35.290922 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:35.297426 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326b038-3716-4d5f-8e99-9697047140e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:35.298885 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326b038-3716-4d5f-8e99-9697047140e0 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:35.299173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326b038-3716-4d5f-8e99-9697047140e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:35.300000 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326b038-3716-4d5f-8e99-9697047140e0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:35.300628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326b038-3716-4d5f-8e99-9697047140e0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:35.302094 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326b038-3716-4d5f-8e99-9697047140e0 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:35.311782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-5326b038-3716-4d5f-8e99-9697047140e0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 22 23:56:35.311782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:35.311782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:56:35.311782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:56:35.311782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:56:35.311782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:56:35.311782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:56:35.311782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:35.311782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 22 23:56:35.311782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:35.311782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:35.311782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:35.311782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:35.311782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:35.311782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:35.311782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:35.311782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:35.312825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:35.312825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:35.312825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:35.312825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:35.312825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:35.312825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:35.312825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:56:35.312825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:56:35.312825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:56:35.312825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:56:35.312825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:35.312825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:35.312825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:35.312825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:35.312825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:35.312825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:35.313679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:35.313679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:35.313679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:35.313679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:35.313679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:35.313679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:35.313679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:35.313679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:35.313679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:35.313679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:35.313679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:56:35.313679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:35.313679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 22 23:56:35.313679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:35.313679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 370/745] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:56:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:35.319657 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f57231e-b81f-4ac4-b37d-853614497380 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:35.328685 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326b038-3716-4d5f-8e99-9697047140e0 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:35.380932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:35.441195 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f57231e-b81f-4ac4-b37d-853614497380 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:35.451626 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326b038-3716-4d5f-8e99-9697047140e0 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:35.484879 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9171896-840f-428e-ae4c-3773aabf82cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:35.486314 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9171896-840f-428e-ae4c-3773aabf82cc None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:35.487578 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9171896-840f-428e-ae4c-3773aabf82cc None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:35.487707 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9171896-840f-428e-ae4c-3773aabf82cc None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:35.488821 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9171896-840f-428e-ae4c-3773aabf82cc None admin] RBAC: Adding query filter params (name=compute) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:35.489491 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9171896-840f-428e-ae4c-3773aabf82cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:35.491143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9171896-840f-428e-ae4c-3773aabf82cc None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:35.493756 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db18421-e782-4145-b67c-66df02e03ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:35.494509 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db18421-e782-4145-b67c-66df02e03ced None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:35.494864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db18421-e782-4145-b67c-66df02e03ced None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:35.495214 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db18421-e782-4145-b67c-66df02e03ced None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:35.495869 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db18421-e782-4145-b67c-66df02e03ced None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:35.496992 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 376/746] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Oct 22 23:56:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:35.502007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db18421-e782-4145-b67c-66df02e03ced None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:35.503583 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9171896-840f-428e-ae4c-3773aabf82cc None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:35.511757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-1db18421-e782-4145-b67c-66df02e03ced None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 22 23:56:35.511757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:35.511757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:56:35.511757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:56:35.511757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:56:35.511757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:56:35.511757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:56:35.511757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:35.511757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 22 23:56:35.511757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:35.511757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:35.511757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:35.511757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:35.511757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:35.511757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:35.511757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:35.511757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:35.512808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:35.512808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:35.512808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:35.512808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:35.512808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:35.512808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:35.512808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:56:35.512808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:56:35.512808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:56:35.512808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:56:35.512808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:35.512808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:35.512808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:35.512808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:35.512808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:35.512808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:35.513678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:35.513678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:35.513678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:35.513678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:35.513678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:35.513678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:35.513678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:35.513678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:35.513678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:35.513678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:35.513678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:56:35.513678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:35.513678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 22 23:56:35.513678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:35.517878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 371/747] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Oct 22 23:56:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:56:35.519132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db18421-e782-4145-b67c-66df02e03ced None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:35.581339 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:35.624363 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db18421-e782-4145-b67c-66df02e03ced None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:35.635072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9171896-840f-428e-ae4c-3773aabf82cc None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:35.647725 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:35.662636 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd02be-7a22-48c7-8dc3-e31356fae380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:35.663834 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd02be-7a22-48c7-8dc3-e31356fae380 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:35.664343 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd02be-7a22-48c7-8dc3-e31356fae380 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:35.665098 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd02be-7a22-48c7-8dc3-e31356fae380 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:35.666364 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd02be-7a22-48c7-8dc3-e31356fae380 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:35.667137 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd02be-7a22-48c7-8dc3-e31356fae380 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:35.669384 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd02be-7a22-48c7-8dc3-e31356fae380 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:35.677945 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 372/748] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Oct 22 23:56:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:35.681187 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4401e8-72d3-49c5-9495-aa63305cb017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:35.682054 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4401e8-72d3-49c5-9495-aa63305cb017 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:35.682549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4401e8-72d3-49c5-9495-aa63305cb017 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:35.683050 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4401e8-72d3-49c5-9495-aa63305cb017 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:35.683348 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd02be-7a22-48c7-8dc3-e31356fae380 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:35.684192 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4401e8-72d3-49c5-9495-aa63305cb017 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:35.684737 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4401e8-72d3-49c5-9495-aa63305cb017 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:35.686613 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4401e8-72d3-49c5-9495-aa63305cb017 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:35.699418 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 377/749] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Oct 22 23:56:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:35.701285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4401e8-72d3-49c5-9495-aa63305cb017 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:35.803735 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd02be-7a22-48c7-8dc3-e31356fae380 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:35.833168 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:35.847508 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328cf1e-6ba1-4d79-95ab-be26620d91a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:35.848538 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328cf1e-6ba1-4d79-95ab-be26620d91a9 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:35.849025 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328cf1e-6ba1-4d79-95ab-be26620d91a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:35.850109 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328cf1e-6ba1-4d79-95ab-be26620d91a9 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:35.850948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328cf1e-6ba1-4d79-95ab-be26620d91a9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:35.851467 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328cf1e-6ba1-4d79-95ab-be26620d91a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:35.853958 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328cf1e-6ba1-4d79-95ab-be26620d91a9 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:35.861421 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 373/750] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Oct 22 23:56:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:35.869267 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328cf1e-6ba1-4d79-95ab-be26620d91a9 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:35.961879 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4401e8-72d3-49c5-9495-aa63305cb017 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:36.002101 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22c80c-1bd7-454a-abfb-9b1919625c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:36.002865 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22c80c-1bd7-454a-abfb-9b1919625c46 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:36.003122 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22c80c-1bd7-454a-abfb-9b1919625c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:36.003403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22c80c-1bd7-454a-abfb-9b1919625c46 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:36.004104 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22c80c-1bd7-454a-abfb-9b1919625c46 None admin] RBAC: Adding query filter params (interface=public, service_id=e99b0a2baeb34901b39189e60c48b060, region_id=RegionOne) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:36.004414 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22c80c-1bd7-454a-abfb-9b1919625c46 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:36.006331 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22c80c-1bd7-454a-abfb-9b1919625c46 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:36.012525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 378/751] 10.222.0.13 () {56 vars in 1251 bytes} [Sat Oct 22 23:56:35 2022] GET /identity/v3/endpoints?service_id=e99b0a2baeb34901b39189e60c48b060&interface=public®ion_id=RegionOne => generated 194 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:36.017902 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328cf1e-6ba1-4d79-95ab-be26620d91a9 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:36.020304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22c80c-1bd7-454a-abfb-9b1919625c46 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:36.055820 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e194b-b155-425f-bd38-96f40748fea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:36.056432 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e194b-b155-425f-bd38-96f40748fea1 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:36.056828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e194b-b155-425f-bd38-96f40748fea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:36.057155 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e194b-b155-425f-bd38-96f40748fea1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:36.057871 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e194b-b155-425f-bd38-96f40748fea1 None admin] RBAC: Adding query filter params (interface=admin, service_id=3a70e1a833bd44e0aa0b813378833846, region_id=RegionOne) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:36.058185 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e194b-b155-425f-bd38-96f40748fea1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:36.059570 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e194b-b155-425f-bd38-96f40748fea1 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:36.069172 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:36.070727 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 374/752] 10.222.0.13 () {56 vars in 1249 bytes} [Sat Oct 22 23:56:35 2022] GET /identity/v3/endpoints?service_id=3a70e1a833bd44e0aa0b813378833846&interface=admin®ion_id=RegionOne => generated 193 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:36.072138 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e194b-b155-425f-bd38-96f40748fea1 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:36.135352 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:36.150265 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22c80c-1bd7-454a-abfb-9b1919625c46 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:36.192952 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544d585-14a7-44cc-843c-71292e1d2fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:36.193584 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544d585-14a7-44cc-843c-71292e1d2fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:36.194125 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544d585-14a7-44cc-843c-71292e1d2fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:36.194403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544d585-14a7-44cc-843c-71292e1d2fb3 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:36.195018 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c544d585-14a7-44cc-843c-71292e1d2fb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:36.196400 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c544d585-14a7-44cc-843c-71292e1d2fb3 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:36.204513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 379/753] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Oct 22 23:56:36 2022] GET /identity/v3/services => generated 1957 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 22 23:56:36.323761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e194b-b155-425f-bd38-96f40748fea1 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:36.349177 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5468888-3ee6-4ff3-b278-f68ef7028faa None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:36.349453 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5468888-3ee6-4ff3-b278-f68ef7028faa None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:36.349734 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5468888-3ee6-4ff3-b278-f68ef7028faa None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:36.351328 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 380/754] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 22 23:56:36.357478 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d399d-c250-4f74-82a8-bed163abb095 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:36.357778 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d399d-c250-4f74-82a8-bed163abb095 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:36.358032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d399d-c250-4f74-82a8-bed163abb095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:36.367618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e7e0e-34bb-44b3-b9c2-101bc71cd8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:36.368585 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e7e0e-34bb-44b3-b9c2-101bc71cd8d9 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:36.369041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e7e0e-34bb-44b3-b9c2-101bc71cd8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:36.369513 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e7e0e-34bb-44b3-b9c2-101bc71cd8d9 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:36.370298 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e7e0e-34bb-44b3-b9c2-101bc71cd8d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:36.372461 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e7e0e-34bb-44b3-b9c2-101bc71cd8d9 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:36.377781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-ab6d399d-c250-4f74-82a8-bed163abb095 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:36.378995 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ab6d399d-c250-4f74-82a8-bed163abb095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:36.383323 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 375/755] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Oct 22 23:56:36 2022] GET /identity/v3/services => generated 1957 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 22 23:56:36.458979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 381/756] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:36 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:36.737061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e7e0e-34bb-44b3-b9c2-101bc71cd8d9 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:36.847257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e7e0e-34bb-44b3-b9c2-101bc71cd8d9 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:36.862661 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:36.884158 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfe5e9-106a-46ff-926e-009f87798410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:36.884807 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfe5e9-106a-46ff-926e-009f87798410 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:36.885087 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfe5e9-106a-46ff-926e-009f87798410 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:36.885357 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedfe5e9-106a-46ff-926e-009f87798410 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:36.885994 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedfe5e9-106a-46ff-926e-009f87798410 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:36.887408 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedfe5e9-106a-46ff-926e-009f87798410 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:36.889944 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: WARNING keystone.server.flask.common [None req-dedfe5e9-106a-46ff-926e-009f87798410 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 22 23:56:36.905321 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.sql.core [None req-dedfe5e9-106a-46ff-926e-009f87798410 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 22 23:56:36.905321 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 22 23:56:36.905321 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [parameters: {'user_id': 'd158eb72cf5f43819d858e891ea00129', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 22 23:56:36.905321 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81380) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 22 23:56:36.924133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-dedfe5e9-106a-46ff-926e-009f87798410 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 22 23:56:36.924133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:36.924133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 22 23:56:36.924133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 22 23:56:36.924133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 22 23:56:36.924133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 22 23:56:36.924133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 22 23:56:36.924133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application result = self._query(query) Oct 22 23:56:36.924133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 22 23:56:36.924133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application conn.query(q) Oct 22 23:56:36.924133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 22 23:56:36.924133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 22 23:56:36.924133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 22 23:56:36.924133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application result.read() Oct 22 23:56:36.924133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 22 23:56:36.925162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 22 23:56:36.925162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 22 23:56:36.925162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 22 23:56:36.925162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 22 23:56:36.925162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 22 23:56:36.925162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 22 23:56:36.925162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 22 23:56:36.925162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 22 23:56:36.925162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:36.925162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 22 23:56:36.925162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:36.925162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:36.925162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 22 23:56:36.925162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 22 23:56:36.925162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 22 23:56:36.925162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 22 23:56:36.925162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 22 23:56:36.925162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application next(self.gen) Oct 22 23:56:36.926146 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 22 23:56:36.926146 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application yield resource Oct 22 23:56:36.926146 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 22 23:56:36.926146 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application next(self.gen) Oct 22 23:56:36.926146 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 22 23:56:36.926146 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self.session.rollback() Oct 22 23:56:36.926146 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 22 23:56:36.926146 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self.force_reraise() Oct 22 23:56:36.926146 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 22 23:56:36.926146 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise self.value Oct 22 23:56:36.926146 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 22 23:56:36.926146 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 22 23:56:36.926146 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 22 23:56:36.926146 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application session.commit() Oct 22 23:56:36.926146 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 22 23:56:36.926146 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 22 23:56:36.926146 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 22 23:56:36.927061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._prepare_impl() Oct 22 23:56:36.927061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 22 23:56:36.927061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self.session.flush() Oct 22 23:56:36.927061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 22 23:56:36.927061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._flush(objects) Oct 22 23:56:36.927061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 22 23:56:36.927061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 22 23:56:36.927061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 22 23:56:36.927061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application compat.raise_( Oct 22 23:56:36.927061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 22 23:56:36.927061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception Oct 22 23:56:36.927061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 22 23:56:36.927061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application flush_context.execute() Oct 22 23:56:36.927061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 22 23:56:36.927061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rec.execute(self) Oct 22 23:56:36.927061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 22 23:56:36.927061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 22 23:56:36.927948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 22 23:56:36.927948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 22 23:56:36.927948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 22 23:56:36.927948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 22 23:56:36.927948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 22 23:56:36.927948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 22 23:56:36.927948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 22 23:56:36.927948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 22 23:56:36.927948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 22 23:56:36.927948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 22 23:56:36.927948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 22 23:56:36.927948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 22 23:56:36.927948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 22 23:56:36.927948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 22 23:56:36.927948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 22 23:56:36.927948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception Oct 22 23:56:36.930064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 22 23:56:36.930064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 22 23:56:36.930064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 22 23:56:36.930064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 22 23:56:36.930064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 22 23:56:36.930064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application result = self._query(query) Oct 22 23:56:36.930064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 22 23:56:36.930064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application conn.query(q) Oct 22 23:56:36.930064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 22 23:56:36.930064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 22 23:56:36.930064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 22 23:56:36.930064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application result.read() Oct 22 23:56:36.930064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 22 23:56:36.930064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 22 23:56:36.930064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 22 23:56:36.930064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 22 23:56:36.932061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 22 23:56:36.932061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 22 23:56:36.932061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 22 23:56:36.932061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 22 23:56:36.932061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 22 23:56:36.932061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 22 23:56:36.932061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd158eb72cf5f43819d858e891ea00129', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 22 23:56:36.932061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 22 23:56:36.932061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:36.932061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:36.932061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:36.932061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:36.932061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:36.932061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:36.932061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:36.938290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:36.938290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:36.938290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:36.938290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:36.938290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:36.938290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:36.938290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:36.938290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 22 23:56:36.938290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 22 23:56:36.938290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:36.938290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:36.938290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:56:36.938290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:36.938290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:56:36.938290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:36.938290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 22 23:56:36.938290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 22 23:56:36.941545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 22 23:56:36.941545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 22 23:56:36.941545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 22 23:56:36.941545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 22 23:56:36.941545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 22 23:56:36.941545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:36.941545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 376/757] 10.222.0.13 () {60 vars in 1131 bytes} [Sat Oct 22 23:56:36 2022] POST /identity/v3/users => generated 160 bytes in 190 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:36.946112 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d399d-c250-4f74-82a8-bed163abb095 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:36.978705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:37.061092 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d399d-c250-4f74-82a8-bed163abb095 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:37.096880 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1bf762-6cf2-4641-85a9-08f2bd9c36c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:37.097513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bf762-6cf2-4641-85a9-08f2bd9c36c7 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:37.097851 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bf762-6cf2-4641-85a9-08f2bd9c36c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:37.098125 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1bf762-6cf2-4641-85a9-08f2bd9c36c7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:37.105603 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1bf762-6cf2-4641-85a9-08f2bd9c36c7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:37.108585 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1bf762-6cf2-4641-85a9-08f2bd9c36c7 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:37.122080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-8d1bf762-6cf2-4641-85a9-08f2bd9c36c7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 22 23:56:37.122080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:37.122080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:37.122080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:37.122080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:37.122080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:37.122080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:37.122080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:37.122080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:37.122080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:37.122080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:37.122080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:37.122080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:56:37.122080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:56:37.122080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:56:37.122080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:56:37.123072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:37.123072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:37.123072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:56:37.123072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:37.123072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:56:37.123072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:37.123072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:37.123072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:37.123072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:37.123072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:37.123072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:37.123072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:37.123072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:37.123072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:37.123072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:37.123072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:37.123072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:37.123072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:37.124075 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:37.124075 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:37.124075 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:56:37.124075 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:56:37.124075 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:56:37.124075 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:56:37.124075 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:56:37.124075 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:56:37.124075 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 22 23:56:37.124075 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:37.124075 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 382/758] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:56:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:37.131879 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfe5e9-106a-46ff-926e-009f87798410 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:37.133782 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23999777-680f-42d2-abed-5aa3c126d496 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:37.134081 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23999777-680f-42d2-abed-5aa3c126d496 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:37.134359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23999777-680f-42d2-abed-5aa3c126d496 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:37.135916 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 383/759] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 22 23:56:37.141101 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c649a-f7b3-4a8e-abe2-54bc6d2b1ade None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:37.141387 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c649a-f7b3-4a8e-abe2-54bc6d2b1ade None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:37.141666 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c649a-f7b3-4a8e-abe2-54bc6d2b1ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:37.161880 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-ed3c649a-f7b3-4a8e-abe2-54bc6d2b1ade None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:37.163090 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ed3c649a-f7b3-4a8e-abe2-54bc6d2b1ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:37.248590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 384/760] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:37 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 22 23:56:37.258359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedfe5e9-106a-46ff-926e-009f87798410 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:37.293493 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17183b8-c04a-4f28-95e9-4169c25dde6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:37.294101 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17183b8-c04a-4f28-95e9-4169c25dde6f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:37.294379 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17183b8-c04a-4f28-95e9-4169c25dde6f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:37.294652 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17183b8-c04a-4f28-95e9-4169c25dde6f None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:37.295348 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17183b8-c04a-4f28-95e9-4169c25dde6f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:37.295648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17183b8-c04a-4f28-95e9-4169c25dde6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:37.297276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17183b8-c04a-4f28-95e9-4169c25dde6f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:37.337840 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 377/761] 10.222.0.13 () {56 vars in 1101 bytes} [Sat Oct 22 23:56:37 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:37.528888 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c649a-f7b3-4a8e-abe2-54bc6d2b1ade None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:37.554067 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:37.637583 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c649a-f7b3-4a8e-abe2-54bc6d2b1ade None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:37.673728 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d7f0b-b0ea-4673-8d09-ee761846206f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:37.674371 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d7f0b-b0ea-4673-8d09-ee761846206f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:37.674710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d7f0b-b0ea-4673-8d09-ee761846206f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:37.674990 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d7f0b-b0ea-4673-8d09-ee761846206f None admin] PATH_INFO: `/v3/services/image` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:37.675575 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d7f0b-b0ea-4673-8d09-ee761846206f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:37.676969 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d7f0b-b0ea-4673-8d09-ee761846206f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:37.686377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-501d7f0b-b0ea-4673-8d09-ee761846206f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 22 23:56:37.686377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:37.686377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:56:37.686377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:56:37.686377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:56:37.686377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:56:37.686377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:56:37.686377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:37.686377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 22 23:56:37.686377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:37.686377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:37.686377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:37.686377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:37.686377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:37.686377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:37.686377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:37.686377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:37.687386 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:37.687386 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:37.687386 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:37.687386 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:37.687386 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:37.687386 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:37.687386 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:56:37.687386 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:56:37.687386 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:56:37.687386 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:56:37.687386 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:37.687386 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:37.687386 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:37.687386 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:37.687386 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:37.687386 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:37.688237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:37.688237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:37.688237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:37.688237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:37.688237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:37.688237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:37.688237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:37.688237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:37.688237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:37.688237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:37.688237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:56:37.688237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:37.688237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 22 23:56:37.688237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:37.688237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 385/762] 10.222.0.13 () {56 vars in 1105 bytes} [Sat Oct 22 23:56:37 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:37.694271 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17183b8-c04a-4f28-95e9-4169c25dde6f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:37.730368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:37.800747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17183b8-c04a-4f28-95e9-4169c25dde6f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:37.835711 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8982f70d-f4b3-43a2-baa7-50aba3b4fedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:37.836313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982f70d-f4b3-43a2-baa7-50aba3b4fedf None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:37.836620 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982f70d-f4b3-43a2-baa7-50aba3b4fedf None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:37.836895 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8982f70d-f4b3-43a2-baa7-50aba3b4fedf None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:37.837514 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8982f70d-f4b3-43a2-baa7-50aba3b4fedf None admin] RBAC: Adding query filter params (name=image) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:37.837814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8982f70d-f4b3-43a2-baa7-50aba3b4fedf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:37.839157 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8982f70d-f4b3-43a2-baa7-50aba3b4fedf None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:37.844112 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 378/763] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:37.850172 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d7f0b-b0ea-4673-8d09-ee761846206f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:37.937768 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:37.955030 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d7f0b-b0ea-4673-8d09-ee761846206f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:37.988329 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a84da-c7f0-40a4-88ec-11eac743ec84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:37.988955 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a84da-c7f0-40a4-88ec-11eac743ec84 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:37.989232 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a84da-c7f0-40a4-88ec-11eac743ec84 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:37.989508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a84da-c7f0-40a4-88ec-11eac743ec84 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:37.990128 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a84da-c7f0-40a4-88ec-11eac743ec84 None admin] RBAC: Adding query filter params (type=image) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:37.990432 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a84da-c7f0-40a4-88ec-11eac743ec84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:37.991773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a84da-c7f0-40a4-88ec-11eac743ec84 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:38.001482 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 386/764] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:38.002971 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8982f70d-f4b3-43a2-baa7-50aba3b4fedf None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:38.110468 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:38.116218 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8982f70d-f4b3-43a2-baa7-50aba3b4fedf None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:38.151187 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d5035-19fa-4098-82f6-2e879fc59b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:38.151794 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d5035-19fa-4098-82f6-2e879fc59b19 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:38.152080 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d5035-19fa-4098-82f6-2e879fc59b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:38.152353 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d5035-19fa-4098-82f6-2e879fc59b19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:38.153074 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d5035-19fa-4098-82f6-2e879fc59b19 None admin] RBAC: Adding query filter params (interface=public, service_id=43e5cb9c6b254b65b6ac24e084fefa2c, region_id=RegionOne) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:38.153381 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d5035-19fa-4098-82f6-2e879fc59b19 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:38.154725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d5035-19fa-4098-82f6-2e879fc59b19 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:38.161237 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 379/765] 10.222.0.13 () {56 vars in 1251 bytes} [Sat Oct 22 23:56:38 2022] GET /identity/v3/endpoints?service_id=43e5cb9c6b254b65b6ac24e084fefa2c&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:38.166735 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a84da-c7f0-40a4-88ec-11eac743ec84 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:38.274452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a84da-c7f0-40a4-88ec-11eac743ec84 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:38.308124 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a57b6-d594-4746-afde-394aa51fe255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:38.308767 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a57b6-d594-4746-afde-394aa51fe255 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:38.309057 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a57b6-d594-4746-afde-394aa51fe255 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:38.309335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a57b6-d594-4746-afde-394aa51fe255 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:38.309936 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3a57b6-d594-4746-afde-394aa51fe255 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:38.311286 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3a57b6-d594-4746-afde-394aa51fe255 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:38.324538 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 387/766] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Oct 22 23:56:38 2022] GET /identity/v3/services => generated 1957 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 22 23:56:38.487053 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5474add1-9f97-4a99-bd70-e9bd299b6f66 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:38.487453 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5474add1-9f97-4a99-bd70-e9bd299b6f66 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:38.487648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5474add1-9f97-4a99-bd70-e9bd299b6f66 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:38.489173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 380/767] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:38.495156 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534674c-1575-4bf4-adcf-c431743a9675 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:38.495444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534674c-1575-4bf4-adcf-c431743a9675 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:38.495720 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534674c-1575-4bf4-adcf-c431743a9675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:38.515072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-0534674c-1575-4bf4-adcf-c431743a9675 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:38.516267 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-0534674c-1575-4bf4-adcf-c431743a9675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:38.527226 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:38.590979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 388/768] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:38 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:38.664230 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd3806-9d8b-42a5-b911-d26e07a84a9f None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:38.664561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd3806-9d8b-42a5-b911-d26e07a84a9f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:38.664832 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd3806-9d8b-42a5-b911-d26e07a84a9f None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:38.666327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 381/769] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:38.671513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19773e-8ab8-41a4-95a2-4f765a7d9e5e None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:38.671812 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19773e-8ab8-41a4-95a2-4f765a7d9e5e None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:38.672089 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19773e-8ab8-41a4-95a2-4f765a7d9e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:38.691300 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-9c19773e-8ab8-41a4-95a2-4f765a7d9e5e None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:38.692505 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9c19773e-8ab8-41a4-95a2-4f765a7d9e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:38.760317 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 389/770] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:38 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:38.862337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd3806-9d8b-42a5-b911-d26e07a84a9f None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:38.964021 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd3806-9d8b-42a5-b911-d26e07a84a9f None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:38.997835 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61879fc-94e9-4ae9-a6a7-ffb0cd035119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:38.998466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61879fc-94e9-4ae9-a6a7-ffb0cd035119 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:38.998748 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61879fc-94e9-4ae9-a6a7-ffb0cd035119 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:38.999023 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61879fc-94e9-4ae9-a6a7-ffb0cd035119 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:38.999602 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61879fc-94e9-4ae9-a6a7-ffb0cd035119 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:39.000986 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61879fc-94e9-4ae9-a6a7-ffb0cd035119 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:39.010394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-b61879fc-94e9-4ae9-a6a7-ffb0cd035119 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 22 23:56:39.010394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:39.010394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:56:39.010394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:56:39.010394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:56:39.010394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:56:39.010394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:56:39.010394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:39.010394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 22 23:56:39.010394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:39.010394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:39.010394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:39.010394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:39.010394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:39.010394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:39.010394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:39.010394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:39.011828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:39.011828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:39.011828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:39.011828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:39.011828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:39.011828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:39.011828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:56:39.011828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:56:39.011828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:56:39.011828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:56:39.011828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:39.011828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:39.011828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:39.011828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:39.011828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:39.011828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:39.013115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:39.013115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:39.013115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:39.013115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:39.013115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:39.013115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:39.013115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:39.013115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:39.013115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:39.013115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:39.013115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:56:39.013115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:39.013115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 22 23:56:39.013115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:39.014203 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 382/771] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:56:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:39.017999 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19773e-8ab8-41a4-95a2-4f765a7d9e5e None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:39.033539 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61879fc-94e9-4ae9-a6a7-ffb0cd035119 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:39.062210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:39.099999 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:39.133369 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19773e-8ab8-41a4-95a2-4f765a7d9e5e None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:39.143929 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61879fc-94e9-4ae9-a6a7-ffb0cd035119 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:39.168209 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75797ab1-924e-4a10-824a-047846464c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:39.168851 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75797ab1-924e-4a10-824a-047846464c77 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:39.169133 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75797ab1-924e-4a10-824a-047846464c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:39.169408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75797ab1-924e-4a10-824a-047846464c77 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:39.170029 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75797ab1-924e-4a10-824a-047846464c77 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:39.170331 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75797ab1-924e-4a10-824a-047846464c77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:39.171672 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75797ab1-924e-4a10-824a-047846464c77 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:39.177337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 390/772] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Oct 22 23:56:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:39.182813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75797ab1-924e-4a10-824a-047846464c77 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:39.183556 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6724747-e989-4b3b-8d93-ac663199450a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:39.184193 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6724747-e989-4b3b-8d93-ac663199450a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:39.184481 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6724747-e989-4b3b-8d93-ac663199450a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:39.184788 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6724747-e989-4b3b-8d93-ac663199450a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:39.185368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6724747-e989-4b3b-8d93-ac663199450a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:39.186734 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6724747-e989-4b3b-8d93-ac663199450a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:39.196141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-d6724747-e989-4b3b-8d93-ac663199450a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 22 23:56:39.196141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:39.196141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:56:39.196141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:56:39.196141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:56:39.196141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:56:39.196141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:56:39.196141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:39.196141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 22 23:56:39.196141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:39.196141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:39.196141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:39.196141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:39.196141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:39.196141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:39.196141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:39.196141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:39.197477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:39.197477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:39.197477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:39.197477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:39.197477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:39.197477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:39.197477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:56:39.197477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:56:39.197477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:56:39.197477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:56:39.197477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:39.197477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:39.197477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:39.197477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:39.197477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:39.197477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:39.198337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:39.198337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:39.198337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:39.198337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:39.198337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:39.198337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:39.198337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:39.198337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:39.198337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:39.198337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:39.198337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:56:39.198337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:39.198337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 22 23:56:39.198337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:39.198337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 383/773] 10.222.0.13 () {56 vars in 1126 bytes} [Sat Oct 22 23:56:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:56:39.203138 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6724747-e989-4b3b-8d93-ac663199450a None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:39.296807 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75797ab1-924e-4a10-824a-047846464c77 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:39.303396 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:39.314933 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6724747-e989-4b3b-8d93-ac663199450a None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:39.337167 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd897fe9-c4eb-4cde-8709-eb290769de35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:39.337783 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd897fe9-c4eb-4cde-8709-eb290769de35 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:39.338152 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd897fe9-c4eb-4cde-8709-eb290769de35 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:39.338344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd897fe9-c4eb-4cde-8709-eb290769de35 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:39.338970 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd897fe9-c4eb-4cde-8709-eb290769de35 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:39.339493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd897fe9-c4eb-4cde-8709-eb290769de35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:39.340663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd897fe9-c4eb-4cde-8709-eb290769de35 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:39.346309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 391/774] 10.222.0.13 () {56 vars in 1112 bytes} [Sat Oct 22 23:56:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:39.351706 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd897fe9-c4eb-4cde-8709-eb290769de35 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:39.352143 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccccf05-3f65-439c-9795-496eebfd2246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:39.352775 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccccf05-3f65-439c-9795-496eebfd2246 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:39.353058 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccccf05-3f65-439c-9795-496eebfd2246 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:39.353334 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccccf05-3f65-439c-9795-496eebfd2246 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:39.353957 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccccf05-3f65-439c-9795-496eebfd2246 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:39.354278 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccccf05-3f65-439c-9795-496eebfd2246 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:39.354489 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:39.355635 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccccf05-3f65-439c-9795-496eebfd2246 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:39.361138 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 384/775] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Oct 22 23:56:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:39.366795 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccccf05-3f65-439c-9795-496eebfd2246 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:39.457927 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd897fe9-c4eb-4cde-8709-eb290769de35 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:39.472819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccccf05-3f65-439c-9795-496eebfd2246 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:39.495109 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac17fe-81a5-4f34-864e-20947978c30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:39.496620 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac17fe-81a5-4f34-864e-20947978c30c None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:39.497151 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac17fe-81a5-4f34-864e-20947978c30c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:39.497601 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac17fe-81a5-4f34-864e-20947978c30c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:39.498569 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac17fe-81a5-4f34-864e-20947978c30c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:39.500130 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac17fe-81a5-4f34-864e-20947978c30c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:39.506738 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600db6b5-e9e3-4f7c-9452-2b87858bf980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:39.507346 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600db6b5-e9e3-4f7c-9452-2b87858bf980 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:39.507660 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600db6b5-e9e3-4f7c-9452-2b87858bf980 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:39.507938 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600db6b5-e9e3-4f7c-9452-2b87858bf980 None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:39.509033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600db6b5-e9e3-4f7c-9452-2b87858bf980 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:39.509033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600db6b5-e9e3-4f7c-9452-2b87858bf980 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:39.510353 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600db6b5-e9e3-4f7c-9452-2b87858bf980 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:39.515109 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 385/776] 10.222.0.13 () {56 vars in 1122 bytes} [Sat Oct 22 23:56:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:39.521543 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600db6b5-e9e3-4f7c-9452-2b87858bf980 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:39.525842 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 392/777] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Oct 22 23:56:39 2022] POST /identity/v3/endpoints => generated 334 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 22 23:56:39.550067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:39.639780 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600db6b5-e9e3-4f7c-9452-2b87858bf980 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:39.683961 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1cd572-ecf6-4eb7-8113-e73e8295b020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:39.684603 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1cd572-ecf6-4eb7-8113-e73e8295b020 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:39.684886 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1cd572-ecf6-4eb7-8113-e73e8295b020 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:39.685160 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1cd572-ecf6-4eb7-8113-e73e8295b020 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:39.685826 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1cd572-ecf6-4eb7-8113-e73e8295b020 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:39.687197 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1cd572-ecf6-4eb7-8113-e73e8295b020 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:39.709121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 386/778] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Oct 22 23:56:39 2022] POST /identity/v3/endpoints => generated 325 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 22 23:56:40.576130 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f685e6-bfba-4afd-b3df-b3ee7192e473 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:40.576681 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f685e6-bfba-4afd-b3df-b3ee7192e473 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:40.576681 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f685e6-bfba-4afd-b3df-b3ee7192e473 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:40.578104 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 393/779] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:40.583784 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac756c-eb27-4ca3-9c76-3c68e7382a70 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:40.584203 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac756c-eb27-4ca3-9c76-3c68e7382a70 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:40.584474 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac756c-eb27-4ca3-9c76-3c68e7382a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:40.603085 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-90ac756c-eb27-4ca3-9c76-3c68e7382a70 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:40.604603 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-90ac756c-eb27-4ca3-9c76-3c68e7382a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:40.674404 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:40.684534 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 387/780] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:40.958835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f685e6-bfba-4afd-b3df-b3ee7192e473 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:41.020130 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:41.065910 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f685e6-bfba-4afd-b3df-b3ee7192e473 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:41.099663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14636d5f-b86d-4996-9b71-2453d6a5484f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:41.100295 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14636d5f-b86d-4996-9b71-2453d6a5484f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:41.100610 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14636d5f-b86d-4996-9b71-2453d6a5484f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:41.100891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14636d5f-b86d-4996-9b71-2453d6a5484f None admin] PATH_INFO: `/v3/services/image` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:41.103874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14636d5f-b86d-4996-9b71-2453d6a5484f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:41.103874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14636d5f-b86d-4996-9b71-2453d6a5484f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:41.116367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-14636d5f-b86d-4996-9b71-2453d6a5484f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 22 23:56:41.116367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:41.116367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:56:41.116367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:56:41.116367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:56:41.116367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:56:41.116367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:56:41.116367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:41.116367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 22 23:56:41.116367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:41.116367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:41.116367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:41.116367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:41.116367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:41.116367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:41.116367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:41.116367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:41.117395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:41.117395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:41.117395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:41.117395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:41.117395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:41.117395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:41.117395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:56:41.117395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:56:41.117395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:56:41.117395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:56:41.117395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:41.117395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:41.117395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:41.117395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:41.117395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:41.117395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:41.118337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:41.118337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:41.118337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:41.118337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:41.118337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:41.118337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:41.118337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:41.118337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:41.118337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:41.118337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:41.118337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:56:41.118337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:56:41.118337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 22 23:56:41.118337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:41.120795 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 394/781] 10.222.0.13 () {56 vars in 1105 bytes} [Sat Oct 22 23:56:40 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:41.123438 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac756c-eb27-4ca3-9c76-3c68e7382a70 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:41.222839 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ac756c-eb27-4ca3-9c76-3c68e7382a70 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:41.255804 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45adbbf3-7008-44ab-930c-f7b44336822e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:41.256405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adbbf3-7008-44ab-930c-f7b44336822e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:41.256713 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adbbf3-7008-44ab-930c-f7b44336822e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:41.256987 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adbbf3-7008-44ab-930c-f7b44336822e None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:41.257612 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45adbbf3-7008-44ab-930c-f7b44336822e None admin] RBAC: Adding query filter params (name=image) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:41.257912 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45adbbf3-7008-44ab-930c-f7b44336822e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:41.259266 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45adbbf3-7008-44ab-930c-f7b44336822e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:41.264199 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 388/782] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:41.270283 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14636d5f-b86d-4996-9b71-2453d6a5484f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:41.370823 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14636d5f-b86d-4996-9b71-2453d6a5484f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:41.393461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:41.405627 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09bf67-1b2d-4b63-bb0e-46c9a2668ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:41.406225 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09bf67-1b2d-4b63-bb0e-46c9a2668ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:41.406503 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09bf67-1b2d-4b63-bb0e-46c9a2668ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:41.406776 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09bf67-1b2d-4b63-bb0e-46c9a2668ce3 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:41.407391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09bf67-1b2d-4b63-bb0e-46c9a2668ce3 None admin] RBAC: Adding query filter params (type=image) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:41.408219 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09bf67-1b2d-4b63-bb0e-46c9a2668ce3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:41.409603 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09bf67-1b2d-4b63-bb0e-46c9a2668ce3 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:41.419367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 395/783] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:41.420395 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45adbbf3-7008-44ab-930c-f7b44336822e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:41.483367 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:41.522615 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45adbbf3-7008-44ab-930c-f7b44336822e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:41.556039 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d4660-75e0-428e-a41f-488e6083d925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:41.556676 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d4660-75e0-428e-a41f-488e6083d925 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:41.557017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d4660-75e0-428e-a41f-488e6083d925 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:41.557297 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d4660-75e0-428e-a41f-488e6083d925 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:41.557968 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d4660-75e0-428e-a41f-488e6083d925 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:41.559311 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d4660-75e0-428e-a41f-488e6083d925 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:41.584513 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 389/784] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Oct 22 23:56:41 2022] POST /identity/v3/endpoints => generated 327 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 22 23:56:41.816226 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fdb6b6-5845-447a-b2cc-348e24d33223 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:41.816755 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fdb6b6-5845-447a-b2cc-348e24d33223 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:41.816859 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fdb6b6-5845-447a-b2cc-348e24d33223 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:41.818192 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 396/785] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:41.823893 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09014a-d746-465b-a058-b5ebf545452b None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:41.824189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09014a-d746-465b-a058-b5ebf545452b None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:41.824467 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09014a-d746-465b-a058-b5ebf545452b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:41.843793 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-3f09014a-d746-465b-a058-b5ebf545452b None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:41.845034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-3f09014a-d746-465b-a058-b5ebf545452b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:41.909206 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5dd4b-93bc-47a5-b30e-d06c85fbfc92 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:41.909490 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5dd4b-93bc-47a5-b30e-d06c85fbfc92 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:41.909768 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5dd4b-93bc-47a5-b30e-d06c85fbfc92 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:41.911228 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 397/786] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 22 23:56:41.916157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0d405-2d5c-4f41-b6d7-8ba67c12ce34 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:41.916448 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0d405-2d5c-4f41-b6d7-8ba67c12ce34 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:41.916770 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0d405-2d5c-4f41-b6d7-8ba67c12ce34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:41.919777 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 390/787] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:41.935410 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-19f0d405-2d5c-4f41-b6d7-8ba67c12ce34 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:41.936634 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-19f0d405-2d5c-4f41-b6d7-8ba67c12ce34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:42.003533 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 398/788] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 22 23:56:42.187522 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09014a-d746-465b-a058-b5ebf545452b None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:42.234739 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:42.278697 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0d405-2d5c-4f41-b6d7-8ba67c12ce34 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:42.295105 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09014a-d746-465b-a058-b5ebf545452b None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:42.331359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5123357-d3ae-4948-8b91-007f238f6a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:42.331987 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5123357-d3ae-4948-8b91-007f238f6a0b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:42.332274 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5123357-d3ae-4948-8b91-007f238f6a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:42.332585 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5123357-d3ae-4948-8b91-007f238f6a0b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:42.343229 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5123357-d3ae-4948-8b91-007f238f6a0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:56:42.348878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-e5123357-d3ae-4948-8b91-007f238f6a0b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 22 23:56:42.348878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:42.348878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:42.348878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:42.348878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:42.348878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:42.348878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:42.348878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:42.348878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:42.348878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:42.348878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:42.348878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:42.348878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:56:42.348878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:56:42.348878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:56:42.348878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:56:42.349818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:56:42.349818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:56:42.349818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:56:42.349818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:56:42.349818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:42.349818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:42.349818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:42.349818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:42.349818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:42.349818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:42.349818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:42.349818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:42.349818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:42.349818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:42.349818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:42.349818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:42.350697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:42.350697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:42.350697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:42.350697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:42.350697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:56:42.350697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:56:42.350697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:56:42.350697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:56:42.350697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:56:42.350697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:56:42.350697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 22 23:56:42.350697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:42.350697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 391/789] 10.222.0.13 () {56 vars in 1120 bytes} [Sat Oct 22 23:56:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:42.355784 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5123357-d3ae-4948-8b91-007f238f6a0b None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:42.386814 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0d405-2d5c-4f41-b6d7-8ba67c12ce34 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:42.403597 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:42.427314 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579b69d-a915-49bc-b8f7-ab717390ebff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:42.428060 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579b69d-a915-49bc-b8f7-ab717390ebff None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:42.428408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579b69d-a915-49bc-b8f7-ab717390ebff None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:42.428777 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579b69d-a915-49bc-b8f7-ab717390ebff None admin] PATH_INFO: `/v3/domains/default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:42.429952 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e579b69d-a915-49bc-b8f7-ab717390ebff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:42.431423 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e579b69d-a915-49bc-b8f7-ab717390ebff None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:42.433214 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 399/790] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:42.439042 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579b69d-a915-49bc-b8f7-ab717390ebff None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:42.459956 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5123357-d3ae-4948-8b91-007f238f6a0b None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:42.467391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:42.496785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823b450-b74d-442e-9a9e-dee7f9b60810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:42.497388 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823b450-b74d-442e-9a9e-dee7f9b60810 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:42.497677 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823b450-b74d-442e-9a9e-dee7f9b60810 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:42.497951 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823b450-b74d-442e-9a9e-dee7f9b60810 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:42.498616 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823b450-b74d-442e-9a9e-dee7f9b60810 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:42.498923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823b450-b74d-442e-9a9e-dee7f9b60810 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:42.500280 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823b450-b74d-442e-9a9e-dee7f9b60810 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:42.512550 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 392/791] 10.222.0.13 () {56 vars in 1115 bytes} [Sat Oct 22 23:56:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:42.514817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823b450-b74d-442e-9a9e-dee7f9b60810 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:42.553584 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579b69d-a915-49bc-b8f7-ab717390ebff None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:42.591928 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9b0da-2124-4e66-894b-4902898bfc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:42.592888 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9b0da-2124-4e66-894b-4902898bfc00 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:42.593316 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9b0da-2124-4e66-894b-4902898bfc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:42.593765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9b0da-2124-4e66-894b-4902898bfc00 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:42.594668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9b0da-2124-4e66-894b-4902898bfc00 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:42.597444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9b0da-2124-4e66-894b-4902898bfc00 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:42.618329 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823b450-b74d-442e-9a9e-dee7f9b60810 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:42.634711 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 400/792] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Oct 22 23:56:42 2022] POST /identity/v3/projects => generated 311 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:56:42.665241 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a23fb-ac6e-4907-98e0-e31967ef5c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:42.665893 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a23fb-ac6e-4907-98e0-e31967ef5c6b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:42.666177 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a23fb-ac6e-4907-98e0-e31967ef5c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:42.666450 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a23fb-ac6e-4907-98e0-e31967ef5c6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:42.679229 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a23fb-ac6e-4907-98e0-e31967ef5c6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:42.680724 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a23fb-ac6e-4907-98e0-e31967ef5c6b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:42.697093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-d00a23fb-ac6e-4907-98e0-e31967ef5c6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:42.697093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:42.697093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:42.697093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:42.697093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:42.697093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:42.697093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:42.697093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:42.697093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:42.697093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:42.697093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:42.697093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:42.697093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:42.697093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:42.697093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:42.697093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:42.698154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:42.698154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:42.698154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:42.698154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:42.698154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:42.698154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:42.698154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:42.698154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:42.698154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:42.698154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:42.698154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:42.698154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:42.698154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:42.698154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:42.698154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:42.698154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:42.698154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:42.699099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:42.699099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:42.699099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:42.700818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 393/793] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:56:42.704211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9b0da-2124-4e66-894b-4902898bfc00 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:42.744397 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:42.819155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9b0da-2124-4e66-894b-4902898bfc00 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:42.856016 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a008c545-56d1-4a13-b493-e30a1956fee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:42.856686 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008c545-56d1-4a13-b493-e30a1956fee5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:42.856974 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008c545-56d1-4a13-b493-e30a1956fee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:42.857252 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008c545-56d1-4a13-b493-e30a1956fee5 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:42.857896 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a008c545-56d1-4a13-b493-e30a1956fee5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:42.858203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a008c545-56d1-4a13-b493-e30a1956fee5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:42.859565 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a008c545-56d1-4a13-b493-e30a1956fee5 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:42.872928 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 401/794] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:42.879450 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a23fb-ac6e-4907-98e0-e31967ef5c6b None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:42.903286 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:42.987890 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a23fb-ac6e-4907-98e0-e31967ef5c6b None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:43.021796 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed110e18-e005-42ba-8982-ebaa610bbc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:43.022417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed110e18-e005-42ba-8982-ebaa610bbc15 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:43.022698 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed110e18-e005-42ba-8982-ebaa610bbc15 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:43.022969 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed110e18-e005-42ba-8982-ebaa610bbc15 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:43.030909 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed110e18-e005-42ba-8982-ebaa610bbc15 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:43.032320 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed110e18-e005-42ba-8982-ebaa610bbc15 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:43.044390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-ed110e18-e005-42ba-8982-ebaa610bbc15 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 22 23:56:43.044390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:43.044390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:43.044390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:43.044390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:43.044390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:43.044390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:43.044390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:43.044390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:43.044390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:43.044390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:43.044390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:43.044390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:56:43.044390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:56:43.044390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:56:43.044390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:56:43.045495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:43.045495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:43.045495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:56:43.045495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:43.045495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:56:43.045495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:43.045495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:43.045495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:43.045495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:43.045495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:43.045495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:43.045495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:43.045495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:43.045495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:43.045495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:43.045495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:43.045495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:43.045495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:43.046459 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:43.046459 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:43.046459 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:56:43.046459 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:56:43.046459 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:56:43.046459 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:56:43.046459 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:56:43.046459 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:56:43.046459 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 22 23:56:43.046459 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:43.047571 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 394/795] 10.222.0.13 () {56 vars in 1093 bytes} [Sat Oct 22 23:56:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:43.051999 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a008c545-56d1-4a13-b493-e30a1956fee5 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:43.157428 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:43.165286 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a008c545-56d1-4a13-b493-e30a1956fee5 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:43.199575 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0863a6-7504-4773-8ba5-8ba9ab883be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:43.200258 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0863a6-7504-4773-8ba5-8ba9ab883be5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:43.200637 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0863a6-7504-4773-8ba5-8ba9ab883be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:43.200973 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0863a6-7504-4773-8ba5-8ba9ab883be5 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:43.201756 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0863a6-7504-4773-8ba5-8ba9ab883be5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:43.202125 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0863a6-7504-4773-8ba5-8ba9ab883be5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:43.203578 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0863a6-7504-4773-8ba5-8ba9ab883be5 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:43.223053 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 402/796] 10.222.0.13 () {56 vars in 1133 bytes} [Sat Oct 22 23:56:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:43.228581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed110e18-e005-42ba-8982-ebaa610bbc15 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:43.310548 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:43.335188 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed110e18-e005-42ba-8982-ebaa610bbc15 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:43.369891 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889553e0-aac4-48e5-b690-ba3505556d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:43.370611 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889553e0-aac4-48e5-b690-ba3505556d63 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:43.370955 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889553e0-aac4-48e5-b690-ba3505556d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:43.371292 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889553e0-aac4-48e5-b690-ba3505556d63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:43.383402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889553e0-aac4-48e5-b690-ba3505556d63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:43.384963 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889553e0-aac4-48e5-b690-ba3505556d63 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:43.399789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-889553e0-aac4-48e5-b690-ba3505556d63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:43.399789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:43.399789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:43.399789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:43.399789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:43.399789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:43.399789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:43.399789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:43.399789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:43.399789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:43.399789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:43.399789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:43.399789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:43.399789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:43.399789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:43.399789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:43.401027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:43.401027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:43.401027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:43.401027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:43.401027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:43.401027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:43.401027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:43.401027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:43.401027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:43.401027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:43.401027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:43.401027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:43.401027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:43.401027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:43.401027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:43.401027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:43.401027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:43.402156 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:43.402156 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:43.402156 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:43.402909 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 395/797] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:56:43.407955 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0863a6-7504-4773-8ba5-8ba9ab883be5 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:43.513667 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0863a6-7504-4773-8ba5-8ba9ab883be5 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:43.548734 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dfd32b-e774-4ffa-b57f-7ad7759588de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:43.549905 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dfd32b-e774-4ffa-b57f-7ad7759588de None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:43.550186 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dfd32b-e774-4ffa-b57f-7ad7759588de None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:43.550515 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dfd32b-e774-4ffa-b57f-7ad7759588de None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:43.551102 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dfd32b-e774-4ffa-b57f-7ad7759588de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:43.551405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dfd32b-e774-4ffa-b57f-7ad7759588de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:43.552806 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dfd32b-e774-4ffa-b57f-7ad7759588de None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:43.571546 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 403/798] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:43.573021 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889553e0-aac4-48e5-b690-ba3505556d63 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:43.674083 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889553e0-aac4-48e5-b690-ba3505556d63 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:43.707665 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:43.713793 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6490e97-196f-49b1-9a81-baa81b0d5fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:43.714439 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6490e97-196f-49b1-9a81-baa81b0d5fcc None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:43.714724 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6490e97-196f-49b1-9a81-baa81b0d5fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:43.714999 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6490e97-196f-49b1-9a81-baa81b0d5fcc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:43.722491 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6490e97-196f-49b1-9a81-baa81b0d5fcc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:43.723919 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6490e97-196f-49b1-9a81-baa81b0d5fcc None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:43.734998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-a6490e97-196f-49b1-9a81-baa81b0d5fcc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:43.734998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:43.734998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:43.734998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:43.734998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:43.734998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:43.734998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:43.734998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:43.734998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:43.734998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:43.734998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:43.734998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:43.734998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:56:43.734998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:56:43.734998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:56:43.735923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:56:43.735923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:43.735923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:43.735923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:43.735923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:43.735923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:43.735923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:43.735923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:43.735923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:43.735923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:43.735923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:43.735923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:43.735923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:43.735923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:43.735923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:43.735923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:43.735923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:43.742017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:56:43.742017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:56:43.742017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:56:43.742017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:56:43.742017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:56:43.742017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:56:43.742017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:43.742017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:43.742017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 396/799] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:56:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:43.743778 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dfd32b-e774-4ffa-b57f-7ad7759588de None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:43.761431 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:43.804673 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070a20c-b6da-4329-ac66-8f48b636d90f None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:43.804988 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070a20c-b6da-4329-ac66-8f48b636d90f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:43.805250 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070a20c-b6da-4329-ac66-8f48b636d90f None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:43.806581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 397/800] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 22 23:56:43.811488 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1439fc9-b194-49cc-bd02-a35d66c5bbdd None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:43.811772 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1439fc9-b194-49cc-bd02-a35d66c5bbdd None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:43.812049 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1439fc9-b194-49cc-bd02-a35d66c5bbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:43.831634 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-d1439fc9-b194-49cc-bd02-a35d66c5bbdd None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:43.832784 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d1439fc9-b194-49cc-bd02-a35d66c5bbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:43.853722 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dfd32b-e774-4ffa-b57f-7ad7759588de None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:43.890212 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d46f19-761e-47f7-b7af-07a8bec027dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:43.890822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d46f19-761e-47f7-b7af-07a8bec027dd None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:43.891110 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d46f19-761e-47f7-b7af-07a8bec027dd None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:43.891391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d46f19-761e-47f7-b7af-07a8bec027dd None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:43.892129 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d46f19-761e-47f7-b7af-07a8bec027dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:43.892438 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d46f19-761e-47f7-b7af-07a8bec027dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:43.894006 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d46f19-761e-47f7-b7af-07a8bec027dd None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:43.902021 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 398/801] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 22 23:56:43.906602 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 404/802] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Oct 22 23:56:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:43.913339 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1439fc9-b194-49cc-bd02-a35d66c5bbdd None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:44.016077 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1439fc9-b194-49cc-bd02-a35d66c5bbdd None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:44.039265 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:44.052379 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941897d6-e112-4615-bd9f-7e54fbabeeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:44.053146 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941897d6-e112-4615-bd9f-7e54fbabeeae None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:44.053433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941897d6-e112-4615-bd9f-7e54fbabeeae None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:44.053714 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941897d6-e112-4615-bd9f-7e54fbabeeae None admin] PATH_INFO: `/v3/projects/5c9246c059754504afee4711d113a3fe/users/c4a3579d56294b9d8ad36d4d174c8526/roles/348c1c2575e44dc7a916de12f69abca6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:44.055867 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941897d6-e112-4615-bd9f-7e54fbabeeae None admin] RBAC: Authorizing `identity:create_grant(project_id=5c9246c059754504afee4711d113a3fe, user_id=c4a3579d56294b9d8ad36d4d174c8526, role_id=348c1c2575e44dc7a916de12f69abca6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:44.057459 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941897d6-e112-4615-bd9f-7e54fbabeeae None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:44.070753 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 399/803] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:56:43 2022] PUT /identity/v3/projects/5c9246c059754504afee4711d113a3fe/users/c4a3579d56294b9d8ad36d4d174c8526/roles/348c1c2575e44dc7a916de12f69abca6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 22 23:56:44.179805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d46f19-761e-47f7-b7af-07a8bec027dd None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:44.262933 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:44.291939 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d46f19-761e-47f7-b7af-07a8bec027dd None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:44.328075 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d92e76-2a2f-4d53-9c28-1fc2745c2a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:44.328726 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d92e76-2a2f-4d53-9c28-1fc2745c2a0e None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:44.329015 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d92e76-2a2f-4d53-9c28-1fc2745c2a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:44.329296 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d92e76-2a2f-4d53-9c28-1fc2745c2a0e None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:44.329989 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d92e76-2a2f-4d53-9c28-1fc2745c2a0e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:44.331342 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d92e76-2a2f-4d53-9c28-1fc2745c2a0e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:44.339183 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.sql.core [None req-24d92e76-2a2f-4d53-9c28-1fc2745c2a0e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 22 23:56:44.339183 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 22 23:56:44.339183 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [parameters: {'id': 'd8c9857eb6f440558a59c200b8f082a5', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 22 23:56:44.339183 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81381) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 22 23:56:44.347469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-24d92e76-2a2f-4d53-9c28-1fc2745c2a0e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 22 23:56:44.347469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:44.347469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 22 23:56:44.347469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 22 23:56:44.347469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 22 23:56:44.347469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 22 23:56:44.347469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 22 23:56:44.347469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result = self._query(query) Oct 22 23:56:44.347469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 22 23:56:44.347469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application conn.query(q) Oct 22 23:56:44.347469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 22 23:56:44.347469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 22 23:56:44.347469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 22 23:56:44.347469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result.read() Oct 22 23:56:44.348405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 22 23:56:44.348405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 22 23:56:44.348405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 22 23:56:44.348405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 22 23:56:44.348405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 22 23:56:44.348405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 22 23:56:44.348405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 22 23:56:44.348405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 22 23:56:44.348405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 22 23:56:44.348405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:44.348405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 22 23:56:44.348405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:44.348405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:44.348405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 22 23:56:44.348405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 22 23:56:44.348405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 22 23:56:44.348405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 22 23:56:44.348405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 22 23:56:44.349409 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application next(self.gen) Oct 22 23:56:44.349409 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 22 23:56:44.349409 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application yield resource Oct 22 23:56:44.349409 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 22 23:56:44.349409 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application next(self.gen) Oct 22 23:56:44.349409 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 22 23:56:44.349409 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.session.rollback() Oct 22 23:56:44.349409 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 22 23:56:44.349409 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.force_reraise() Oct 22 23:56:44.349409 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 22 23:56:44.349409 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise self.value Oct 22 23:56:44.349409 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 22 23:56:44.349409 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 22 23:56:44.349409 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 22 23:56:44.349409 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application session.commit() Oct 22 23:56:44.349409 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 22 23:56:44.349409 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 22 23:56:44.350338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 22 23:56:44.350338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._prepare_impl() Oct 22 23:56:44.350338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 22 23:56:44.350338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.session.flush() Oct 22 23:56:44.350338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 22 23:56:44.350338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._flush(objects) Oct 22 23:56:44.350338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 22 23:56:44.350338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 22 23:56:44.350338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 22 23:56:44.350338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application compat.raise_( Oct 22 23:56:44.350338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 22 23:56:44.350338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception Oct 22 23:56:44.350338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 22 23:56:44.350338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application flush_context.execute() Oct 22 23:56:44.350338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 22 23:56:44.350338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rec.execute(self) Oct 22 23:56:44.350338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 22 23:56:44.351274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 22 23:56:44.351274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 22 23:56:44.351274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 22 23:56:44.351274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 22 23:56:44.351274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 22 23:56:44.351274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 22 23:56:44.351274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 22 23:56:44.351274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 22 23:56:44.351274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 22 23:56:44.351274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 22 23:56:44.351274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 22 23:56:44.351274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 22 23:56:44.351274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 22 23:56:44.351274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 22 23:56:44.351274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 22 23:56:44.352113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 22 23:56:44.352113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception Oct 22 23:56:44.352113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 22 23:56:44.352113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 22 23:56:44.352113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 22 23:56:44.352113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 22 23:56:44.352113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 22 23:56:44.352113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result = self._query(query) Oct 22 23:56:44.352113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 22 23:56:44.352113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application conn.query(q) Oct 22 23:56:44.352113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 22 23:56:44.352113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 22 23:56:44.352113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 22 23:56:44.352113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application result.read() Oct 22 23:56:44.352113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 22 23:56:44.352113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 22 23:56:44.352113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 22 23:56:44.354805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 22 23:56:44.354805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 22 23:56:44.354805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 22 23:56:44.354805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 22 23:56:44.354805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 22 23:56:44.354805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 22 23:56:44.354805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 22 23:56:44.354805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [parameters: {'id': 'd8c9857eb6f440558a59c200b8f082a5', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 22 23:56:44.354805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 22 23:56:44.354805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:44.354805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:44.354805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:44.354805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:44.354805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 22 23:56:44.354805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 22 23:56:44.357056 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 22 23:56:44.357056 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 22 23:56:44.357056 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 22 23:56:44.357056 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:44.357056 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:56:44.357056 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:44.357056 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:44.357056 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:44.357056 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:44.357056 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:44.357056 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:44.357056 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:44.357056 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:44.357056 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:44.357056 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:44.357056 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:44.357056 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:44.359433 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 22 23:56:44.359433 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 22 23:56:44.359433 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:44.359433 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:44.359433 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 22 23:56:44.359433 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 22 23:56:44.359433 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:44.359433 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:44.359433 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 22 23:56:44.359433 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 22 23:56:44.359433 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 22 23:56:44.359433 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:44.359433 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 405/804] 10.222.0.13 () {60 vars in 1137 bytes} [Sat Oct 22 23:56:44 2022] POST /identity/v3/domains => generated 190 bytes in 170 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:44.361358 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941897d6-e112-4615-bd9f-7e54fbabeeae None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:44.461637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941897d6-e112-4615-bd9f-7e54fbabeeae None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:44.495546 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b8ffc-191a-4a07-bfcb-b3cdfdd07170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:44.496328 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b8ffc-191a-4a07-bfcb-b3cdfdd07170 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:44.496748 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b8ffc-191a-4a07-bfcb-b3cdfdd07170 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:44.497141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b8ffc-191a-4a07-bfcb-b3cdfdd07170 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:44.508626 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b8ffc-191a-4a07-bfcb-b3cdfdd07170 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:44.510189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b8ffc-191a-4a07-bfcb-b3cdfdd07170 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:44.525041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-b04b8ffc-191a-4a07-bfcb-b3cdfdd07170 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:44.525041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:44.525041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:44.525041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:44.525041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:44.525041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:44.525041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:44.525041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:44.525041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:44.525041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:44.525041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:44.525041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:44.525041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:44.525041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:44.525041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:44.525041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:44.526242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:44.526242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:44.526242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:44.526242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:44.526242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:44.526242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:44.526242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:44.526242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:44.526242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:44.526242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:44.526242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:44.526242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:44.526242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:44.526242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:44.526242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:44.526242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:44.526242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:44.527356 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:44.527356 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:44.527356 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:44.528629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 400/805] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:44.535037 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d92e76-2a2f-4d53-9c28-1fc2745c2a0e None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:44.640620 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d92e76-2a2f-4d53-9c28-1fc2745c2a0e None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:44.676204 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f40eb5-a8a1-4093-ba44-3f606e88021f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:44.676969 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f40eb5-a8a1-4093-ba44-3f606e88021f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:44.677257 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f40eb5-a8a1-4093-ba44-3f606e88021f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:44.677534 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f40eb5-a8a1-4093-ba44-3f606e88021f None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:44.678171 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f40eb5-a8a1-4093-ba44-3f606e88021f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:44.678475 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f40eb5-a8a1-4093-ba44-3f606e88021f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:44.679827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f40eb5-a8a1-4093-ba44-3f606e88021f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:44.688009 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:44.695245 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 406/806] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:44.853026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a5b73-d9c1-4c74-a61b-93e5412f4a34 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:44.853549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a5b73-d9c1-4c74-a61b-93e5412f4a34 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:44.853627 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a5b73-d9c1-4c74-a61b-93e5412f4a34 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:44.855092 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 401/807] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 22 23:56:44.860635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd99aa-cb30-4448-9f19-ce78a32040cc None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:44.861581 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd99aa-cb30-4448-9f19-ce78a32040cc None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:44.861766 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd99aa-cb30-4448-9f19-ce78a32040cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:44.881945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-18cd99aa-cb30-4448-9f19-ce78a32040cc None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:44.883141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-18cd99aa-cb30-4448-9f19-ce78a32040cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:44.953231 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 407/808] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:45.227409 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a5b73-d9c1-4c74-a61b-93e5412f4a34 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:45.262327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:45.331067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a5b73-d9c1-4c74-a61b-93e5412f4a34 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:45.364336 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de88d40-476c-4e1a-bdd4-3d024c1f612a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:45.364986 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de88d40-476c-4e1a-bdd4-3d024c1f612a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:45.365267 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de88d40-476c-4e1a-bdd4-3d024c1f612a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:45.365540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de88d40-476c-4e1a-bdd4-3d024c1f612a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:45.366635 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de88d40-476c-4e1a-bdd4-3d024c1f612a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:45.368036 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de88d40-476c-4e1a-bdd4-3d024c1f612a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:45.369620 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 402/809] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:45.375240 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd99aa-cb30-4448-9f19-ce78a32040cc None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:45.474495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd99aa-cb30-4448-9f19-ce78a32040cc None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:45.500237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:45.508984 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c4011-1669-4961-a206-9605c0a33bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:45.509584 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c4011-1669-4961-a206-9605c0a33bfd None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:45.509862 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c4011-1669-4961-a206-9605c0a33bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:45.510131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c4011-1669-4961-a206-9605c0a33bfd None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:45.510767 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c4011-1669-4961-a206-9605c0a33bfd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:45.512204 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c4011-1669-4961-a206-9605c0a33bfd None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:45.536814 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 408/810] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Oct 22 23:56:45 2022] POST /identity/v3/users => generated 285 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:56:46.272255 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bad7b8-bb16-48a0-ba6a-50136f50562d None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:46.272877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bad7b8-bb16-48a0-ba6a-50136f50562d None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:46.272877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bad7b8-bb16-48a0-ba6a-50136f50562d None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:46.274187 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 403/811] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:46.279629 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60749054-f8ca-4fde-8bb8-8969ccbef193 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:46.280035 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60749054-f8ca-4fde-8bb8-8969ccbef193 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:46.280183 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60749054-f8ca-4fde-8bb8-8969ccbef193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:46.299148 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-60749054-f8ca-4fde-8bb8-8969ccbef193 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:46.300329 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-60749054-f8ca-4fde-8bb8-8969ccbef193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:46.365237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 409/812] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:46.631866 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bad7b8-bb16-48a0-ba6a-50136f50562d None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:46.732305 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bad7b8-bb16-48a0-ba6a-50136f50562d None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:46.733811 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:46.769929 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9d95a-d44b-49ee-9ffe-8730a8379064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:46.770551 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9d95a-d44b-49ee-9ffe-8730a8379064 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:46.770829 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9d95a-d44b-49ee-9ffe-8730a8379064 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:46.771100 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9d95a-d44b-49ee-9ffe-8730a8379064 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:46.780896 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9d95a-d44b-49ee-9ffe-8730a8379064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:56:46.786211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-f2f9d95a-d44b-49ee-9ffe-8730a8379064 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 22 23:56:46.786211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:46.786211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:46.786211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:46.786211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:46.786211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:46.786211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:46.786211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:46.786211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:46.786211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:46.786211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:46.786211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:46.786211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:56:46.786211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:56:46.786211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:56:46.786211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:56:46.787372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:56:46.787372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:56:46.787372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:56:46.787372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:56:46.787372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:46.787372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:46.787372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:46.787372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:46.787372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:46.787372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:46.787372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:46.787372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:46.787372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:46.787372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:46.787372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:46.787372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:46.788380 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:46.788380 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:46.788380 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:46.788380 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:46.788380 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:56:46.788380 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:56:46.788380 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:56:46.788380 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:56:46.788380 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:56:46.788380 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:56:46.788380 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 22 23:56:46.788380 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:46.788380 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 404/813] 10.222.0.13 () {56 vars in 1120 bytes} [Sat Oct 22 23:56:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:46.793377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60749054-f8ca-4fde-8bb8-8969ccbef193 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:46.890327 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60749054-f8ca-4fde-8bb8-8969ccbef193 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:46.903849 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:46.925655 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb099e37-975c-4b70-8e88-c9bebff3bdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:46.926266 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb099e37-975c-4b70-8e88-c9bebff3bdf8 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:46.926553 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb099e37-975c-4b70-8e88-c9bebff3bdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:46.926831 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb099e37-975c-4b70-8e88-c9bebff3bdf8 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:46.927484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb099e37-975c-4b70-8e88-c9bebff3bdf8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:46.927790 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb099e37-975c-4b70-8e88-c9bebff3bdf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:46.929171 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb099e37-975c-4b70-8e88-c9bebff3bdf8 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:46.937103 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 410/814] 10.222.0.13 () {56 vars in 1115 bytes} [Sat Oct 22 23:56:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:46.943030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9d95a-d44b-49ee-9ffe-8730a8379064 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:47.039990 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9d95a-d44b-49ee-9ffe-8730a8379064 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:47.072612 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60eb6e-1185-4e9e-884b-d007ddac6c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:47.073228 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60eb6e-1185-4e9e-884b-d007ddac6c39 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:47.073512 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60eb6e-1185-4e9e-884b-d007ddac6c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:47.073792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60eb6e-1185-4e9e-884b-d007ddac6c39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:47.084180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60eb6e-1185-4e9e-884b-d007ddac6c39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:47.085648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60eb6e-1185-4e9e-884b-d007ddac6c39 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:47.099823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-7e60eb6e-1185-4e9e-884b-d007ddac6c39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:47.099823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:47.099823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:47.099823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:47.099823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:47.099823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:47.099823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:47.099823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:47.099823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:47.099823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:47.099823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:47.099823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:47.099823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:47.099823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:47.099823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:47.099823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:47.100884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:47.100884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:47.100884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:47.100884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:47.100884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:47.100884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:47.100884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:47.100884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:47.100884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:47.100884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:47.100884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:47.100884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:47.100884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:47.100884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:47.100884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:47.100884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:47.100884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:47.102335 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:47.102335 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:47.102335 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:47.102335 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 405/815] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:56:47.106597 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb099e37-975c-4b70-8e88-c9bebff3bdf8 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:47.205525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb099e37-975c-4b70-8e88-c9bebff3bdf8 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:47.238475 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00590e2a-4716-4e01-8901-2f7b6d3769a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:47.239081 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00590e2a-4716-4e01-8901-2f7b6d3769a7 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:47.239365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00590e2a-4716-4e01-8901-2f7b6d3769a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:47.239641 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00590e2a-4716-4e01-8901-2f7b6d3769a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:47.240271 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00590e2a-4716-4e01-8901-2f7b6d3769a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:47.240610 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00590e2a-4716-4e01-8901-2f7b6d3769a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:47.241963 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00590e2a-4716-4e01-8901-2f7b6d3769a7 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:47.254298 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 411/816] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:47.260007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60eb6e-1185-4e9e-884b-d007ddac6c39 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:47.275858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:47.362384 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60eb6e-1185-4e9e-884b-d007ddac6c39 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:47.395172 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3545b5-31f3-423a-b6e5-c13530531603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:47.395794 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3545b5-31f3-423a-b6e5-c13530531603 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:47.396074 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3545b5-31f3-423a-b6e5-c13530531603 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:47.396350 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3545b5-31f3-423a-b6e5-c13530531603 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:47.403169 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3545b5-31f3-423a-b6e5-c13530531603 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:47.404634 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3545b5-31f3-423a-b6e5-c13530531603 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:47.416451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-ff3545b5-31f3-423a-b6e5-c13530531603 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 22 23:56:47.416451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:47.416451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:47.416451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:47.416451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:47.416451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:47.416451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:47.416451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:47.416451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:47.416451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:47.416451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:47.416451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:47.416451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:56:47.416451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:56:47.416451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:56:47.416451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:56:47.417497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:47.417497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:47.417497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:56:47.417497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:47.417497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:56:47.417497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:56:47.417497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:47.417497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:47.417497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:47.417497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:47.417497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:47.417497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:47.417497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:47.417497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:47.417497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:47.417497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:47.417497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:47.417497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:47.418471 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:47.418471 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:47.418471 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:56:47.418471 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:56:47.418471 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:56:47.418471 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:56:47.418471 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:56:47.418471 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:56:47.418471 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 22 23:56:47.418471 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:47.418471 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 406/817] 10.222.0.13 () {56 vars in 1093 bytes} [Sat Oct 22 23:56:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:47.423920 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00590e2a-4716-4e01-8901-2f7b6d3769a7 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:47.458303 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:47.539587 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00590e2a-4716-4e01-8901-2f7b6d3769a7 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:47.573217 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947387a-ae64-4817-b07d-fb2868d95334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:47.573809 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947387a-ae64-4817-b07d-fb2868d95334 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:47.574088 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947387a-ae64-4817-b07d-fb2868d95334 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:47.574359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947387a-ae64-4817-b07d-fb2868d95334 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:47.575060 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947387a-ae64-4817-b07d-fb2868d95334 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:47.575359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947387a-ae64-4817-b07d-fb2868d95334 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:47.576786 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947387a-ae64-4817-b07d-fb2868d95334 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:47.593569 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 412/818] 10.222.0.13 () {56 vars in 1133 bytes} [Sat Oct 22 23:56:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:47.600572 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3545b5-31f3-423a-b6e5-c13530531603 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:47.678664 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:47.703840 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3545b5-31f3-423a-b6e5-c13530531603 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:47.729906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980abc5-a623-49b6-a334-d9c0da474df3 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:47.730120 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980abc5-a623-49b6-a334-d9c0da474df3 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:47.730403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7980abc5-a623-49b6-a334-d9c0da474df3 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:47.731733 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 413/819] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 22 23:56:47.736980 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7eeb3-4ef6-4347-bfb1-975d19e8b398 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:47.737262 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7eeb3-4ef6-4347-bfb1-975d19e8b398 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:47.737536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7eeb3-4ef6-4347-bfb1-975d19e8b398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:47.738376 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc94b7-60d4-4fdc-857e-8f1bb1bce0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:47.738984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc94b7-60d4-4fdc-857e-8f1bb1bce0ed None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:47.739270 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc94b7-60d4-4fdc-857e-8f1bb1bce0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:47.739548 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc94b7-60d4-4fdc-857e-8f1bb1bce0ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:47.754879 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-b0a7eeb3-4ef6-4347-bfb1-975d19e8b398 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:47.756085 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b0a7eeb3-4ef6-4347-bfb1-975d19e8b398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:47.807193 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc94b7-60d4-4fdc-857e-8f1bb1bce0ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:47.808672 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc94b7-60d4-4fdc-857e-8f1bb1bce0ed None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:47.822674 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 414/820] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 22 23:56:47.879952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:47.905708 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 407/821] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:56:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 22 23:56:47.912060 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7eeb3-4ef6-4347-bfb1-975d19e8b398 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:47.936145 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:48.008613 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7eeb3-4ef6-4347-bfb1-975d19e8b398 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:48.041821 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e77a27-e353-4729-a8ca-5ae21c46c898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:48.042445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e77a27-e353-4729-a8ca-5ae21c46c898 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:48.042728 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e77a27-e353-4729-a8ca-5ae21c46c898 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:48.043051 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e77a27-e353-4729-a8ca-5ae21c46c898 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:48.052902 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e77a27-e353-4729-a8ca-5ae21c46c898 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:48.054328 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e77a27-e353-4729-a8ca-5ae21c46c898 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:48.069026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-88e77a27-e353-4729-a8ca-5ae21c46c898 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:48.069026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:48.069026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:48.069026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:48.069026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:48.069026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:48.069026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:48.069026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:48.069026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:48.069026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:48.069026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:48.069026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:48.069026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:56:48.069026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:56:48.069026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:56:48.069026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:56:48.074895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:48.074895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:48.074895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:48.074895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:48.074895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:48.074895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:48.074895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:48.074895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:48.074895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:48.074895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:48.074895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:48.074895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:48.074895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:48.074895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:48.074895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:48.074895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:48.074895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:56:48.075860 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:56:48.075860 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:56:48.075860 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:48.075860 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 415/822] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:48.076108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc94b7-60d4-4fdc-857e-8f1bb1bce0ed None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:48.093638 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e77a27-e353-4729-a8ca-5ae21c46c898 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:48.173795 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:48.178423 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc94b7-60d4-4fdc-857e-8f1bb1bce0ed None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:48.197529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e77a27-e353-4729-a8ca-5ae21c46c898 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:48.212552 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591a87e-8e9b-4fd6-a750-b3be7c3797cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:48.213221 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591a87e-8e9b-4fd6-a750-b3be7c3797cc None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:48.213532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591a87e-8e9b-4fd6-a750-b3be7c3797cc None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:48.213841 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591a87e-8e9b-4fd6-a750-b3be7c3797cc None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:48.214556 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3591a87e-8e9b-4fd6-a750-b3be7c3797cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:48.214887 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3591a87e-8e9b-4fd6-a750-b3be7c3797cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:48.216364 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3591a87e-8e9b-4fd6-a750-b3be7c3797cc None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:48.227426 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 408/823] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:48.230620 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86010db-351e-4cc2-bb5c-ca63c48dc2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:48.231245 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86010db-351e-4cc2-bb5c-ca63c48dc2fc None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:48.231529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86010db-351e-4cc2-bb5c-ca63c48dc2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:48.231804 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86010db-351e-4cc2-bb5c-ca63c48dc2fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:48.233095 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591a87e-8e9b-4fd6-a750-b3be7c3797cc None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:48.242182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86010db-351e-4cc2-bb5c-ca63c48dc2fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:56:48.247465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-b86010db-351e-4cc2-bb5c-ca63c48dc2fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:56:48.247465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:48.247465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:48.247465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:48.247465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:48.247465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:48.247465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:48.247465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:48.247465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:48.247465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:48.247465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:48.247465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:48.247465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:56:48.247465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:56:48.247465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:56:48.247465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:56:48.248461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:56:48.248461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:56:48.248461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:56:48.248461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:56:48.248461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:48.248461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:48.248461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:48.248461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:48.248461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:48.248461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:48.248461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:48.248461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:48.248461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:48.248461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:48.248461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:48.248461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:48.249369 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:48.249369 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:48.249369 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:48.249369 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:48.249369 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:56:48.249369 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:56:48.249369 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:56:48.249369 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:56:48.249369 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:56:48.249369 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:56:48.249369 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:56:48.249369 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:48.249369 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 416/824] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Oct 22 23:56:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:56:48.255396 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86010db-351e-4cc2-bb5c-ca63c48dc2fc None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:48.268351 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:48.334536 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591a87e-8e9b-4fd6-a750-b3be7c3797cc None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:48.356008 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86010db-351e-4cc2-bb5c-ca63c48dc2fc None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:48.367066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7182e-c0ad-4669-b426-5073dbb6ce95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:48.367691 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7182e-c0ad-4669-b426-5073dbb6ce95 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:48.367973 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7182e-c0ad-4669-b426-5073dbb6ce95 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:48.368249 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7182e-c0ad-4669-b426-5073dbb6ce95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:48.374677 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee7182e-c0ad-4669-b426-5073dbb6ce95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:48.376099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee7182e-c0ad-4669-b426-5073dbb6ce95 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:48.388446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-3ee7182e-c0ad-4669-b426-5073dbb6ce95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:48.388446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:48.388446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:48.388446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:48.388446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:48.388446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:48.388446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:48.388446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:48.388446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:48.388446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:48.388446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:48.388446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:48.388446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:56:48.388446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:56:48.388446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:56:48.389428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:56:48.389428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:48.389428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:48.389428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:48.389428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:48.389428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:48.389428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:48.389428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:48.389428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:48.389428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:48.389428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:48.389428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:48.389428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:48.389428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:48.389428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:48.389428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:48.389428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:48.390366 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:56:48.390366 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:56:48.390366 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:56:48.390366 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:56:48.390366 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:56:48.390366 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:56:48.390366 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:56:48.390366 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:56:48.390842 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28d871-12d9-4a03-8a32-74a35ac3cd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:48.390939 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 409/825] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:56:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:56:48.391024 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28d871-12d9-4a03-8a32-74a35ac3cd8c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:48.391024 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28d871-12d9-4a03-8a32-74a35ac3cd8c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:48.391024 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28d871-12d9-4a03-8a32-74a35ac3cd8c None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:48.391024 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28d871-12d9-4a03-8a32-74a35ac3cd8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:48.391303 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28d871-12d9-4a03-8a32-74a35ac3cd8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:48.393365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28d871-12d9-4a03-8a32-74a35ac3cd8c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:48.395919 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:48.396089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7182e-c0ad-4669-b426-5073dbb6ce95 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:48.403202 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 417/826] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:56:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:48.408971 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28d871-12d9-4a03-8a32-74a35ac3cd8c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:48.493782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:48.500849 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7182e-c0ad-4669-b426-5073dbb6ce95 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:48.511966 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28d871-12d9-4a03-8a32-74a35ac3cd8c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:48.535566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539e1f8-49da-4db7-a5dd-5adc43ce57f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:48.536197 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539e1f8-49da-4db7-a5dd-5adc43ce57f5 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:48.536478 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539e1f8-49da-4db7-a5dd-5adc43ce57f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:48.536779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539e1f8-49da-4db7-a5dd-5adc43ce57f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:48.537466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539e1f8-49da-4db7-a5dd-5adc43ce57f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:48.537769 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539e1f8-49da-4db7-a5dd-5adc43ce57f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:48.539179 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539e1f8-49da-4db7-a5dd-5adc43ce57f5 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:48.546369 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca575a-c8da-449e-9dda-856ea5d8893c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:48.546993 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca575a-c8da-449e-9dda-856ea5d8893c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:48.547279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca575a-c8da-449e-9dda-856ea5d8893c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:48.547556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca575a-c8da-449e-9dda-856ea5d8893c None admin] PATH_INFO: `/v3/users/74211264f7734bf1bb44d5bcb119df0f` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:48.551468 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 410/827] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Oct 22 23:56:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:48.557442 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539e1f8-49da-4db7-a5dd-5adc43ce57f5 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:48.561551 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca575a-c8da-449e-9dda-856ea5d8893c None admin] RBAC: Authorizing `identity:get_user(user_id=74211264f7734bf1bb44d5bcb119df0f)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:48.562972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca575a-c8da-449e-9dda-856ea5d8893c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:48.569798 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 418/828] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:56:48 2022] GET /identity/v3/users/74211264f7734bf1bb44d5bcb119df0f => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:48.570700 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca575a-c8da-449e-9dda-856ea5d8893c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:48.627449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:48.655325 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539e1f8-49da-4db7-a5dd-5adc43ce57f5 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:48.671095 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca575a-c8da-449e-9dda-856ea5d8893c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:48.691834 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca5a1c-416b-4d19-82a9-9e07acfba5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:48.692451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca5a1c-416b-4d19-82a9-9e07acfba5ae None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:48.692779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca5a1c-416b-4d19-82a9-9e07acfba5ae None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:48.693054 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca5a1c-416b-4d19-82a9-9e07acfba5ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:48.693773 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca5a1c-416b-4d19-82a9-9e07acfba5ae None admin] RBAC: Adding query filter params (role.id=348c1c2575e44dc7a916de12f69abca6, scope.project.id=5c9246c059754504afee4711d113a3fe, user.id=c4a3579d56294b9d8ad36d4d174c8526) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:48.694085 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca5a1c-416b-4d19-82a9-9e07acfba5ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:48.695763 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca5a1c-416b-4d19-82a9-9e07acfba5ae None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:48.701549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 411/829] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:56:48 2022] GET /identity/v3/role_assignments?user.id=c4a3579d56294b9d8ad36d4d174c8526&scope.project.id=5c9246c059754504afee4711d113a3fe&role.id=348c1c2575e44dc7a916de12f69abca6 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:56:48.706587 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83209e8d-e8fb-4ab8-a354-dfc209495438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:48.707256 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83209e8d-e8fb-4ab8-a354-dfc209495438 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:48.707525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83209e8d-e8fb-4ab8-a354-dfc209495438 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:48.707802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83209e8d-e8fb-4ab8-a354-dfc209495438 None admin] PATH_INFO: `/v3/projects/51c9d09dbef54c67acb233c5499df98b` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:48.709001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83209e8d-e8fb-4ab8-a354-dfc209495438 None admin] RBAC: Authorizing `identity:get_project(project_id=51c9d09dbef54c67acb233c5499df98b)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:48.710458 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83209e8d-e8fb-4ab8-a354-dfc209495438 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:48.712176 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 419/830] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:56:48 2022] GET /identity/v3/projects/51c9d09dbef54c67acb233c5499df98b => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:48.718374 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca5a1c-416b-4d19-82a9-9e07acfba5ae None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:48.739269 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:48.824829 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca5a1c-416b-4d19-82a9-9e07acfba5ae None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:48.862728 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a2242-6d38-47d2-83ae-ae055ad9db83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:48.863398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a2242-6d38-47d2-83ae-ae055ad9db83 None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:48.863679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a2242-6d38-47d2-83ae-ae055ad9db83 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:48.863953 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a2242-6d38-47d2-83ae-ae055ad9db83 None admin] PATH_INFO: `/v3/projects/51c9d09dbef54c67acb233c5499df98b/users/74211264f7734bf1bb44d5bcb119df0f/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:48.865905 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a2242-6d38-47d2-83ae-ae055ad9db83 None admin] RBAC: Authorizing `identity:create_grant(project_id=51c9d09dbef54c67acb233c5499df98b, user_id=74211264f7734bf1bb44d5bcb119df0f, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:48.867315 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a2242-6d38-47d2-83ae-ae055ad9db83 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:48.880961 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 412/831] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:56:48 2022] PUT /identity/v3/projects/51c9d09dbef54c67acb233c5499df98b/users/74211264f7734bf1bb44d5bcb119df0f/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:56:51.030034 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e05ac3-001f-490f-9634-1410cac93229 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:51.030611 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e05ac3-001f-490f-9634-1410cac93229 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:51.030611 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e05ac3-001f-490f-9634-1410cac93229 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:51.031910 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 420/832] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:51.037228 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e551607-a7b8-420d-8255-c591c7fcffa0 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:51.037643 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e551607-a7b8-420d-8255-c591c7fcffa0 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:51.037781 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e551607-a7b8-420d-8255-c591c7fcffa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:51.056091 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-2e551607-a7b8-420d-8255-c591c7fcffa0 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:51.057301 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2e551607-a7b8-420d-8255-c591c7fcffa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:51.122057 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 413/833] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:51.387282 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e05ac3-001f-490f-9634-1410cac93229 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:51.486439 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e05ac3-001f-490f-9634-1410cac93229 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:51.509835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:51.521418 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e4666-b32a-4186-866a-03ccf0e3dac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:51.522045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e4666-b32a-4186-866a-03ccf0e3dac8 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:51.522330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e4666-b32a-4186-866a-03ccf0e3dac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:51.522608 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e4666-b32a-4186-866a-03ccf0e3dac8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:51.532444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e4666-b32a-4186-866a-03ccf0e3dac8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:56:51.538028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-638e4666-b32a-4186-866a-03ccf0e3dac8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:56:51.538028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:56:51.538028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:56:51.538028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:56:51.538028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:56:51.538028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:56:51.538028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:56:51.538028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:56:51.538028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:56:51.538028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:56:51.538028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:56:51.538028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:56:51.538028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:56:51.538028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:56:51.538028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:56:51.538028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:56:51.539136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:56:51.539136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:56:51.539136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:56:51.539136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:56:51.539136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:56:51.539136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:56:51.539136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:56:51.539136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:56:51.539136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:56:51.539136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:56:51.539136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:56:51.539136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:56:51.539136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:56:51.539136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:56:51.539136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:56:51.539136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:56:51.540180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:56:51.540180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:56:51.540180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:56:51.540180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:56:51.540180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:56:51.540180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:56:51.540180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:56:51.540180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:56:51.540180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:56:51.540180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:56:51.540180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:56:51.540180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:56:51.540180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 421/834] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Oct 22 23:56:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:56:51.545326 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e551607-a7b8-420d-8255-c591c7fcffa0 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:51.555163 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:51.641657 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e551607-a7b8-420d-8255-c591c7fcffa0 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:51.674966 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df338db-7bb2-46cb-8022-2f099b16b816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:51.675646 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df338db-7bb2-46cb-8022-2f099b16b816 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:51.675883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df338db-7bb2-46cb-8022-2f099b16b816 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:51.676107 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df338db-7bb2-46cb-8022-2f099b16b816 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:51.676797 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df338db-7bb2-46cb-8022-2f099b16b816 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:51.677210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df338db-7bb2-46cb-8022-2f099b16b816 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:51.678454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df338db-7bb2-46cb-8022-2f099b16b816 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:51.686390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 414/835] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:56:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:51.692252 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e4666-b32a-4186-866a-03ccf0e3dac8 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:51.790271 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e4666-b32a-4186-866a-03ccf0e3dac8 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:51.823178 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711b926-56c5-409c-b042-7026942c7428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:51.823790 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711b926-56c5-409c-b042-7026942c7428 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:51.824074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711b926-56c5-409c-b042-7026942c7428 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:51.824351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711b926-56c5-409c-b042-7026942c7428 None admin] PATH_INFO: `/v3/users/74211264f7734bf1bb44d5bcb119df0f` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:51.825496 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711b926-56c5-409c-b042-7026942c7428 None admin] RBAC: Authorizing `identity:get_user(user_id=74211264f7734bf1bb44d5bcb119df0f)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:51.826905 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711b926-56c5-409c-b042-7026942c7428 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:51.828445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 422/836] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:56:51 2022] GET /identity/v3/users/74211264f7734bf1bb44d5bcb119df0f => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:51.834113 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df338db-7bb2-46cb-8022-2f099b16b816 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:51.901030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:51.932309 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df338db-7bb2-46cb-8022-2f099b16b816 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:51.964914 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9fb5d-c72b-4901-a106-854f7021d6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:51.965502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9fb5d-c72b-4901-a106-854f7021d6fa None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:51.965780 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9fb5d-c72b-4901-a106-854f7021d6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:51.966052 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9fb5d-c72b-4901-a106-854f7021d6fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:52.031124 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f9fb5d-c72b-4901-a106-854f7021d6fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:52.032709 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f9fb5d-c72b-4901-a106-854f7021d6fa None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:52.098738 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:52.130941 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 415/837] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:56:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 22 23:56:52.136891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711b926-56c5-409c-b042-7026942c7428 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:52.188713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:52.234668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711b926-56c5-409c-b042-7026942c7428 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:52.266901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af884f3-42e4-4763-9f3e-56572e983207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:52.268040 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af884f3-42e4-4763-9f3e-56572e983207 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:52.268430 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af884f3-42e4-4763-9f3e-56572e983207 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:52.268627 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af884f3-42e4-4763-9f3e-56572e983207 None admin] PATH_INFO: `/v3/projects/51c9d09dbef54c67acb233c5499df98b` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:52.269745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af884f3-42e4-4763-9f3e-56572e983207 None admin] RBAC: Authorizing `identity:get_project(project_id=51c9d09dbef54c67acb233c5499df98b)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:52.271165 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af884f3-42e4-4763-9f3e-56572e983207 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:52.272801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 423/838] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:56:52 2022] GET /identity/v3/projects/51c9d09dbef54c67acb233c5499df98b => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:52.278869 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9fb5d-c72b-4901-a106-854f7021d6fa None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:52.374445 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9fb5d-c72b-4901-a106-854f7021d6fa None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:52.406780 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a42ef3-1b6b-42f7-84ed-fbf083e544e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:52.407398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a42ef3-1b6b-42f7-84ed-fbf083e544e0 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:52.407677 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a42ef3-1b6b-42f7-84ed-fbf083e544e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:52.408056 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a42ef3-1b6b-42f7-84ed-fbf083e544e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:52.408869 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a42ef3-1b6b-42f7-84ed-fbf083e544e0 None admin] RBAC: Adding query filter params (role.id=839a55263e2c4cbaa2f945b75cf298d6, scope.project.id=51c9d09dbef54c67acb233c5499df98b, user.id=74211264f7734bf1bb44d5bcb119df0f) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:56:52.409178 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a42ef3-1b6b-42f7-84ed-fbf083e544e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:52.410574 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a42ef3-1b6b-42f7-84ed-fbf083e544e0 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:52.416107 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 416/839] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:56:52 2022] GET /identity/v3/role_assignments?user.id=74211264f7734bf1bb44d5bcb119df0f&scope.project.id=51c9d09dbef54c67acb233c5499df98b&role.id=839a55263e2c4cbaa2f945b75cf298d6 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:54.537140 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afd24a-6e87-4b21-a811-dd309f82ad26 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:54.537654 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afd24a-6e87-4b21-a811-dd309f82ad26 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:54.537654 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afd24a-6e87-4b21-a811-dd309f82ad26 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:54.538922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 424/840] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:54.544080 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f4a12-efee-4173-aca1-4c374b9c93a1 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:54.544458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f4a12-efee-4173-aca1-4c374b9c93a1 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:54.544661 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f4a12-efee-4173-aca1-4c374b9c93a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:54.562432 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-6e9f4a12-efee-4173-aca1-4c374b9c93a1 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:54.563619 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-6e9f4a12-efee-4173-aca1-4c374b9c93a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:54.589491 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:54.627934 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 417/841] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 22 23:56:54.902791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afd24a-6e87-4b21-a811-dd309f82ad26 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:55.003078 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afd24a-6e87-4b21-a811-dd309f82ad26 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:55.022204 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:55.036736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22e642-2d7d-45b5-ad45-94516c645814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:55.037347 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22e642-2d7d-45b5-ad45-94516c645814 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:55.037626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22e642-2d7d-45b5-ad45-94516c645814 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:55.037900 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22e642-2d7d-45b5-ad45-94516c645814 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:55.038979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e22e642-2d7d-45b5-ad45-94516c645814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:55.040379 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e22e642-2d7d-45b5-ad45-94516c645814 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:55.041931 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 425/842] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:56:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:55.047719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f4a12-efee-4173-aca1-4c374b9c93a1 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:55.146940 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f4a12-efee-4173-aca1-4c374b9c93a1 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:55.180694 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6797a-5d4d-457f-82a5-0a4da69b1d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:55.181279 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6797a-5d4d-457f-82a5-0a4da69b1d83 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:55.181579 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6797a-5d4d-457f-82a5-0a4da69b1d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:55.181892 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6797a-5d4d-457f-82a5-0a4da69b1d83 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:55.182603 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6797a-5d4d-457f-82a5-0a4da69b1d83 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:55.184063 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6797a-5d4d-457f-82a5-0a4da69b1d83 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:55.204502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 418/843] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Oct 22 23:56:55 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:56:57.349233 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd1c28-1d67-4e41-b203-8834f94c549c None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:57.349753 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd1c28-1d67-4e41-b203-8834f94c549c None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:57.349874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd1c28-1d67-4e41-b203-8834f94c549c None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:57.351115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 426/844] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:56:57.356224 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643aa6cd-165f-40d9-8862-bc26e3db4add None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:57.356641 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643aa6cd-165f-40d9-8862-bc26e3db4add None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:57.356816 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643aa6cd-165f-40d9-8862-bc26e3db4add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:57.374663 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-643aa6cd-165f-40d9-8862-bc26e3db4add None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:56:57.375778 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-643aa6cd-165f-40d9-8862-bc26e3db4add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:56:57.389840 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:57.442610 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 419/845] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:56:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:56:57.710225 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd1c28-1d67-4e41-b203-8834f94c549c None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:57.807607 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd1c28-1d67-4e41-b203-8834f94c549c None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:57.839506 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc2d86-587e-4b9c-9103-bd0641d886a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:57.840116 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc2d86-587e-4b9c-9103-bd0641d886a5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:57.840475 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc2d86-587e-4b9c-9103-bd0641d886a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:57.840778 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc2d86-587e-4b9c-9103-bd0641d886a5 None admin] PATH_INFO: `/v3/roles/cf8116e4e82748ac872d8763d113ba61` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:57.842131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bc2d86-587e-4b9c-9103-bd0641d886a5 None admin] RBAC: Authorizing `identity:get_role(role_id=cf8116e4e82748ac872d8763d113ba61)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:57.843466 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bc2d86-587e-4b9c-9103-bd0641d886a5 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:57.844783 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 427/846] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:56:57 2022] GET /identity/v3/roles/cf8116e4e82748ac872d8763d113ba61 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:57.851155 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643aa6cd-165f-40d9-8862-bc26e3db4add None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:57.947482 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643aa6cd-165f-40d9-8862-bc26e3db4add None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:57.979904 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1bcee-1796-4c1c-98e7-f190758d1cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:57.980461 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1bcee-1796-4c1c-98e7-f190758d1cae None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:57.980770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1bcee-1796-4c1c-98e7-f190758d1cae None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:57.981044 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1bcee-1796-4c1c-98e7-f190758d1cae None admin] PATH_INFO: `/v3/users/75846fa6b0364b49935625a1718053bc` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:57.994102 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c1bcee-1796-4c1c-98e7-f190758d1cae None admin] RBAC: Authorizing `identity:get_user(user_id=75846fa6b0364b49935625a1718053bc)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:57.995512 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c1bcee-1796-4c1c-98e7-f190758d1cae None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:57.997104 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 420/847] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:56:57 2022] GET /identity/v3/users/75846fa6b0364b49935625a1718053bc => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:58.002669 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc2d86-587e-4b9c-9103-bd0641d886a5 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:58.056541 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:58.098797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc2d86-587e-4b9c-9103-bd0641d886a5 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:58.131234 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48963223-b71e-49da-a3f8-18a116a007af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:58.131868 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48963223-b71e-49da-a3f8-18a116a007af None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:58.132150 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48963223-b71e-49da-a3f8-18a116a007af None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:58.132420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48963223-b71e-49da-a3f8-18a116a007af None admin] PATH_INFO: `/v3/projects/51c9d09dbef54c67acb233c5499df98b` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:58.133541 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48963223-b71e-49da-a3f8-18a116a007af None admin] RBAC: Authorizing `identity:get_project(project_id=51c9d09dbef54c67acb233c5499df98b)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:58.134938 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48963223-b71e-49da-a3f8-18a116a007af None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:58.136554 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 428/848] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:56:58 2022] GET /identity/v3/projects/51c9d09dbef54c67acb233c5499df98b => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:56:58.142387 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1bcee-1796-4c1c-98e7-f190758d1cae None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:56:58.153232 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:56:58.240745 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1bcee-1796-4c1c-98e7-f190758d1cae None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:56:58.273331 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3a2cc-73d0-47c2-ba7f-dfd891893e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:56:58.273973 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3a2cc-73d0-47c2-ba7f-dfd891893e4e None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:56:58.274253 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3a2cc-73d0-47c2-ba7f-dfd891893e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:56:58.274529 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3a2cc-73d0-47c2-ba7f-dfd891893e4e None admin] PATH_INFO: `/v3/projects/51c9d09dbef54c67acb233c5499df98b/users/75846fa6b0364b49935625a1718053bc/roles/cf8116e4e82748ac872d8763d113ba61` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:56:58.276675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3a2cc-73d0-47c2-ba7f-dfd891893e4e None admin] RBAC: Authorizing `identity:create_grant(project_id=51c9d09dbef54c67acb233c5499df98b, user_id=75846fa6b0364b49935625a1718053bc, role_id=cf8116e4e82748ac872d8763d113ba61)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:56:58.278098 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3a2cc-73d0-47c2-ba7f-dfd891893e4e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:56:58.291401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 421/849] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:56:58 2022] PUT /identity/v3/projects/51c9d09dbef54c67acb233c5499df98b/users/75846fa6b0364b49935625a1718053bc/roles/cf8116e4e82748ac872d8763d113ba61 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:57:00.411673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf90ab-399c-41b0-af77-5208eeb31250 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:00.412194 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf90ab-399c-41b0-af77-5208eeb31250 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:00.412194 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf90ab-399c-41b0-af77-5208eeb31250 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:00.413498 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 429/850] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:57:00.418732 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93f3ee-4cdf-4e8a-86bb-15bc03692e54 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:00.419150 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93f3ee-4cdf-4e8a-86bb-15bc03692e54 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:00.419285 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93f3ee-4cdf-4e8a-86bb-15bc03692e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:00.437426 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-5a93f3ee-4cdf-4e8a-86bb-15bc03692e54 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:57:00.438635 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-5a93f3ee-4cdf-4e8a-86bb-15bc03692e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:57:00.505244 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:00.511360 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 422/851] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:57:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:57:00.780378 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf90ab-399c-41b0-af77-5208eeb31250 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:00.879008 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf90ab-399c-41b0-af77-5208eeb31250 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:00.901870 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:00.913327 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df9575-72ea-4ed7-ab76-e17e9e93016e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:00.913942 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df9575-72ea-4ed7-ab76-e17e9e93016e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:00.914224 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df9575-72ea-4ed7-ab76-e17e9e93016e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:00.914499 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df9575-72ea-4ed7-ab76-e17e9e93016e None admin] PATH_INFO: `/v3/roles/cf8116e4e82748ac872d8763d113ba61` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:00.915861 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df9575-72ea-4ed7-ab76-e17e9e93016e None admin] RBAC: Authorizing `identity:get_role(role_id=cf8116e4e82748ac872d8763d113ba61)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:00.917249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df9575-72ea-4ed7-ab76-e17e9e93016e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:00.918546 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 430/852] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:57:00 2022] GET /identity/v3/roles/cf8116e4e82748ac872d8763d113ba61 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:57:00.925514 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93f3ee-4cdf-4e8a-86bb-15bc03692e54 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:01.020741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93f3ee-4cdf-4e8a-86bb-15bc03692e54 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:01.053852 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964da59b-992d-43ff-abfd-11587fc22f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:01.054413 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964da59b-992d-43ff-abfd-11587fc22f3d None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:01.054693 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964da59b-992d-43ff-abfd-11587fc22f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:01.054968 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964da59b-992d-43ff-abfd-11587fc22f3d None admin] PATH_INFO: `/v3/users/75846fa6b0364b49935625a1718053bc` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:01.056072 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964da59b-992d-43ff-abfd-11587fc22f3d None admin] RBAC: Authorizing `identity:get_user(user_id=75846fa6b0364b49935625a1718053bc)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:01.057526 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964da59b-992d-43ff-abfd-11587fc22f3d None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:01.059078 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 423/853] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:57:00 2022] GET /identity/v3/users/75846fa6b0364b49935625a1718053bc => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:57:01.065270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df9575-72ea-4ed7-ab76-e17e9e93016e None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:01.163234 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df9575-72ea-4ed7-ab76-e17e9e93016e None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:01.195530 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d2112-2661-4313-933d-e903ce2c28cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:01.196126 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d2112-2661-4313-933d-e903ce2c28cd None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:01.196408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d2112-2661-4313-933d-e903ce2c28cd None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:01.196710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d2112-2661-4313-933d-e903ce2c28cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:01.251807 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:01.261197 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9d2112-2661-4313-933d-e903ce2c28cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:01.262619 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9d2112-2661-4313-933d-e903ce2c28cd None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:01.356425 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 431/854] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:57:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 22 23:57:01.362952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964da59b-992d-43ff-abfd-11587fc22f3d None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:01.425746 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:01.460537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964da59b-992d-43ff-abfd-11587fc22f3d None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:01.492820 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c12edf-990f-4cf8-a123-a3de992cbc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:01.493441 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c12edf-990f-4cf8-a123-a3de992cbc9b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:01.493726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c12edf-990f-4cf8-a123-a3de992cbc9b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:01.494006 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c12edf-990f-4cf8-a123-a3de992cbc9b None admin] PATH_INFO: `/v3/projects/51c9d09dbef54c67acb233c5499df98b` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:01.495116 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c12edf-990f-4cf8-a123-a3de992cbc9b None admin] RBAC: Authorizing `identity:get_project(project_id=51c9d09dbef54c67acb233c5499df98b)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:01.496560 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c12edf-990f-4cf8-a123-a3de992cbc9b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:01.498171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 424/855] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:57:01 2022] GET /identity/v3/projects/51c9d09dbef54c67acb233c5499df98b => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:57:01.505055 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d2112-2661-4313-933d-e903ce2c28cd None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:01.591545 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:01.601171 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d2112-2661-4313-933d-e903ce2c28cd None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:01.634096 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971febd-9d9b-417a-b70f-996fc4055bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:01.634627 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971febd-9d9b-417a-b70f-996fc4055bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:01.634907 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971febd-9d9b-417a-b70f-996fc4055bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:01.635180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971febd-9d9b-417a-b70f-996fc4055bf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:01.635882 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f971febd-9d9b-417a-b70f-996fc4055bf9 None admin] RBAC: Adding query filter params (role.id=cf8116e4e82748ac872d8763d113ba61, scope.project.id=51c9d09dbef54c67acb233c5499df98b, user.id=75846fa6b0364b49935625a1718053bc) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:57:01.636188 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f971febd-9d9b-417a-b70f-996fc4055bf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:01.637619 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f971febd-9d9b-417a-b70f-996fc4055bf9 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:01.643205 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 432/856] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:57:01 2022] GET /identity/v3/role_assignments?user.id=75846fa6b0364b49935625a1718053bc&scope.project.id=51c9d09dbef54c67acb233c5499df98b&role.id=cf8116e4e82748ac872d8763d113ba61 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:57:03.767046 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2557451-feb2-4114-9cd0-c44b63cf9c7a None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:03.767646 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2557451-feb2-4114-9cd0-c44b63cf9c7a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:03.767646 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2557451-feb2-4114-9cd0-c44b63cf9c7a None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:03.769000 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 425/857] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:57:03.774081 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccbde1-342e-4be9-8619-785d44d84c38 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:03.774497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccbde1-342e-4be9-8619-785d44d84c38 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:03.774629 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccbde1-342e-4be9-8619-785d44d84c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:03.792505 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-72ccbde1-342e-4be9-8619-785d44d84c38 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:57:03.793683 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-72ccbde1-342e-4be9-8619-785d44d84c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:57:03.857443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 433/858] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:57:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:57:04.121833 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2557451-feb2-4114-9cd0-c44b63cf9c7a None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:04.222656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2557451-feb2-4114-9cd0-c44b63cf9c7a None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:04.255090 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab65e0f-7b1f-46fd-8479-61bfd9d5a0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:04.255702 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab65e0f-7b1f-46fd-8479-61bfd9d5a0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:04.255981 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab65e0f-7b1f-46fd-8479-61bfd9d5a0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:04.256253 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab65e0f-7b1f-46fd-8479-61bfd9d5a0c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:04.257358 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab65e0f-7b1f-46fd-8479-61bfd9d5a0c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:04.258802 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab65e0f-7b1f-46fd-8479-61bfd9d5a0c2 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:04.260326 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 426/859] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:57:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:57:04.265987 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccbde1-342e-4be9-8619-785d44d84c38 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:04.347009 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:04.361613 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccbde1-342e-4be9-8619-785d44d84c38 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:04.393965 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccae660-d5bd-44a7-9fa9-eb97dd3c1f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:04.394566 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccae660-d5bd-44a7-9fa9-eb97dd3c1f93 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:04.394846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccae660-d5bd-44a7-9fa9-eb97dd3c1f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:04.395116 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccae660-d5bd-44a7-9fa9-eb97dd3c1f93 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:04.395751 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccae660-d5bd-44a7-9fa9-eb97dd3c1f93 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:04.397178 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccae660-d5bd-44a7-9fa9-eb97dd3c1f93 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:04.422361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 434/860] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Oct 22 23:57:04 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:57:06.578706 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a2225-7a06-4d4b-b5cb-7540a1f7bc2c None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:06.579213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a2225-7a06-4d4b-b5cb-7540a1f7bc2c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:06.579213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a2225-7a06-4d4b-b5cb-7540a1f7bc2c None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:06.581274 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 427/861] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:57:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:57:06.586459 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e21de7-224a-4db7-adb4-162048c43abc None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:06.586890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e21de7-224a-4db7-adb4-162048c43abc None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:06.587014 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e21de7-224a-4db7-adb4-162048c43abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:06.605785 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-24e21de7-224a-4db7-adb4-162048c43abc None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:57:06.606950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-24e21de7-224a-4db7-adb4-162048c43abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:57:06.672054 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 435/862] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:57:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:57:06.942720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a2225-7a06-4d4b-b5cb-7540a1f7bc2c None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:06.945868 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:07.042975 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a2225-7a06-4d4b-b5cb-7540a1f7bc2c None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:07.076145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8cf0c8-34de-44e6-a66b-1a2d2876ab66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:07.076789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cf0c8-34de-44e6-a66b-1a2d2876ab66 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:07.077074 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cf0c8-34de-44e6-a66b-1a2d2876ab66 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:07.077351 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cf0c8-34de-44e6-a66b-1a2d2876ab66 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:07.078443 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8cf0c8-34de-44e6-a66b-1a2d2876ab66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:07.079853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8cf0c8-34de-44e6-a66b-1a2d2876ab66 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:07.082008 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 428/863] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:57:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:57:07.087697 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e21de7-224a-4db7-adb4-162048c43abc None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:07.153831 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:07.187089 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e21de7-224a-4db7-adb4-162048c43abc None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:07.221227 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c188ec-f1c0-4c4a-b3b7-a78b6443c506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:07.221909 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c188ec-f1c0-4c4a-b3b7-a78b6443c506 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:07.222194 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c188ec-f1c0-4c4a-b3b7-a78b6443c506 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:07.222471 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c188ec-f1c0-4c4a-b3b7-a78b6443c506 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:07.223121 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c188ec-f1c0-4c4a-b3b7-a78b6443c506 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:07.224548 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c188ec-f1c0-4c4a-b3b7-a78b6443c506 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:07.243934 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 436/864] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Oct 22 23:57:07 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 22 23:57:09.403351 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45719b-cb48-42ef-abd9-e287e6c97e3c None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:09.403863 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45719b-cb48-42ef-abd9-e287e6c97e3c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:09.403863 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45719b-cb48-42ef-abd9-e287e6c97e3c None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:09.405204 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 429/865] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:57:09.410978 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7e419-06fc-451f-b341-e87d5326d933 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:09.411363 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7e419-06fc-451f-b341-e87d5326d933 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:09.411543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7e419-06fc-451f-b341-e87d5326d933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:09.429898 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-05e7e419-06fc-451f-b341-e87d5326d933 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:57:09.431090 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-05e7e419-06fc-451f-b341-e87d5326d933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:57:09.495884 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 437/866] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:57:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:57:09.769345 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45719b-cb48-42ef-abd9-e287e6c97e3c None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:09.837813 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:09.869955 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45719b-cb48-42ef-abd9-e287e6c97e3c None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:09.902390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43803154-5401-4edb-a031-9e2d54ba40a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:09.903005 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43803154-5401-4edb-a031-9e2d54ba40a1 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:09.903291 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43803154-5401-4edb-a031-9e2d54ba40a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:09.903569 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43803154-5401-4edb-a031-9e2d54ba40a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:09.914090 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43803154-5401-4edb-a031-9e2d54ba40a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:57:09.918935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-43803154-5401-4edb-a031-9e2d54ba40a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:57:09.918935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:57:09.918935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:57:09.918935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:57:09.918935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:57:09.918935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:57:09.918935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:57:09.918935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:57:09.918935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:57:09.918935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:57:09.918935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:57:09.918935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:57:09.918935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:57:09.918935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:57:09.918935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:57:09.918935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:57:09.920121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:57:09.920121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:57:09.920121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:57:09.920121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:57:09.920121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:57:09.920121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:57:09.920121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:57:09.920121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:57:09.920121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:57:09.920121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:57:09.920121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:57:09.920121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:57:09.920121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:57:09.920121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:57:09.920121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:57:09.920121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:57:09.921175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:57:09.921175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:57:09.921175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:57:09.921175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:57:09.921175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:57:09.921175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:57:09.921175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:57:09.921175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:57:09.921175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:57:09.921175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:57:09.921175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:57:09.921175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:57:09.921175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 430/867] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Oct 22 23:57:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:57:09.926048 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7e419-06fc-451f-b341-e87d5326d933 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:09.979437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:10.023952 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7e419-06fc-451f-b341-e87d5326d933 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:10.055947 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed83859-233a-4748-8aec-6ad6ed6171a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:10.056568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed83859-233a-4748-8aec-6ad6ed6171a1 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:10.056856 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed83859-233a-4748-8aec-6ad6ed6171a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:10.057132 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed83859-233a-4748-8aec-6ad6ed6171a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:10.057785 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed83859-233a-4748-8aec-6ad6ed6171a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:57:10.058090 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed83859-233a-4748-8aec-6ad6ed6171a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:10.059439 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed83859-233a-4748-8aec-6ad6ed6171a1 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:10.067648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 438/868] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:57:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:57:10.073612 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43803154-5401-4edb-a031-9e2d54ba40a1 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:10.170243 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43803154-5401-4edb-a031-9e2d54ba40a1 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:10.198158 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:10.204544 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f76fc8-9208-452b-889d-aff17c8ea911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:10.205177 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f76fc8-9208-452b-889d-aff17c8ea911 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:10.205461 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f76fc8-9208-452b-889d-aff17c8ea911 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:10.205740 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f76fc8-9208-452b-889d-aff17c8ea911 None admin] PATH_INFO: `/v3/users/5655b57378a9439fb0b7d71ebed7de25` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:10.218421 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f76fc8-9208-452b-889d-aff17c8ea911 None admin] RBAC: Authorizing `identity:get_user(user_id=5655b57378a9439fb0b7d71ebed7de25)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:10.219853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f76fc8-9208-452b-889d-aff17c8ea911 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:10.221513 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 431/869] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:57:10 2022] GET /identity/v3/users/5655b57378a9439fb0b7d71ebed7de25 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:57:10.227195 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed83859-233a-4748-8aec-6ad6ed6171a1 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:10.324319 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed83859-233a-4748-8aec-6ad6ed6171a1 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:10.339436 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:10.357171 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e1d98-a86c-4999-9bfa-4c03fc6333fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:10.357791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e1d98-a86c-4999-9bfa-4c03fc6333fb None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:10.358076 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e1d98-a86c-4999-9bfa-4c03fc6333fb None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:10.358353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e1d98-a86c-4999-9bfa-4c03fc6333fb None admin] PATH_INFO: `/v3/projects/b50f052204cd4b22aeddbbaf73af76f6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:10.359466 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e1d98-a86c-4999-9bfa-4c03fc6333fb None admin] RBAC: Authorizing `identity:get_project(project_id=b50f052204cd4b22aeddbbaf73af76f6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:10.360919 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e1d98-a86c-4999-9bfa-4c03fc6333fb None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:10.363367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 439/870] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:57:10 2022] GET /identity/v3/projects/b50f052204cd4b22aeddbbaf73af76f6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:57:10.369268 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f76fc8-9208-452b-889d-aff17c8ea911 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:10.471240 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f76fc8-9208-452b-889d-aff17c8ea911 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:10.505151 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6acce3-7634-4bec-bb47-9a88f5dd6059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:10.505796 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6acce3-7634-4bec-bb47-9a88f5dd6059 None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:10.506079 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6acce3-7634-4bec-bb47-9a88f5dd6059 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:10.506356 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6acce3-7634-4bec-bb47-9a88f5dd6059 None admin] PATH_INFO: `/v3/projects/b50f052204cd4b22aeddbbaf73af76f6/users/5655b57378a9439fb0b7d71ebed7de25/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:10.508219 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6acce3-7634-4bec-bb47-9a88f5dd6059 None admin] RBAC: Authorizing `identity:create_grant(project_id=b50f052204cd4b22aeddbbaf73af76f6, user_id=5655b57378a9439fb0b7d71ebed7de25, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:10.509665 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6acce3-7634-4bec-bb47-9a88f5dd6059 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:10.521860 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 432/871] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:57:10 2022] PUT /identity/v3/projects/b50f052204cd4b22aeddbbaf73af76f6/users/5655b57378a9439fb0b7d71ebed7de25/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:57:12.669746 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4dfed-f746-401d-bb8c-f206900afc6a None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:12.670293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4dfed-f746-401d-bb8c-f206900afc6a None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:12.670293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4dfed-f746-401d-bb8c-f206900afc6a None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:12.671641 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 440/872] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:57:12.677796 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ac10f-1f94-4850-af9d-7eb33f462e44 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:12.678222 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ac10f-1f94-4850-af9d-7eb33f462e44 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:12.678337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ac10f-1f94-4850-af9d-7eb33f462e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:12.697226 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-e72ac10f-1f94-4850-af9d-7eb33f462e44 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:57:12.698385 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-e72ac10f-1f94-4850-af9d-7eb33f462e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:57:12.740864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:12.763901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 433/873] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:57:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:57:13.032465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4dfed-f746-401d-bb8c-f206900afc6a None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:13.132253 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4dfed-f746-401d-bb8c-f206900afc6a None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:13.165032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ca7d4-bf82-4415-b5cd-d8237aec94d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:13.165647 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ca7d4-bf82-4415-b5cd-d8237aec94d0 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:13.165927 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ca7d4-bf82-4415-b5cd-d8237aec94d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:13.166197 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ca7d4-bf82-4415-b5cd-d8237aec94d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:13.176718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ca7d4-bf82-4415-b5cd-d8237aec94d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:57:13.181445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-436ca7d4-bf82-4415-b5cd-d8237aec94d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:57:13.181445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:57:13.181445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:57:13.181445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:57:13.181445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:57:13.181445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:57:13.181445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:57:13.181445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:57:13.181445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:57:13.181445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:57:13.181445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:57:13.181445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:57:13.181445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:57:13.181445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:57:13.181445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:57:13.181445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:57:13.182616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:57:13.182616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:57:13.182616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:57:13.182616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:57:13.182616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:57:13.182616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:57:13.182616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:57:13.182616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:57:13.182616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:57:13.182616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:57:13.182616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:57:13.182616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:57:13.182616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:57:13.182616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:57:13.182616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:57:13.182616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:57:13.183682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:57:13.183682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:57:13.183682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:57:13.183682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:57:13.183682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:57:13.183682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:57:13.183682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:57:13.183682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:57:13.183682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:57:13.183682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:57:13.183682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:57:13.183682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:57:13.183682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 441/874] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Oct 22 23:57:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:57:13.188716 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ac10f-1f94-4850-af9d-7eb33f462e44 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:13.284673 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ac10f-1f94-4850-af9d-7eb33f462e44 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:13.316775 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54dd90-d3eb-4744-b12e-be53f571cea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:13.317368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54dd90-d3eb-4744-b12e-be53f571cea2 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:13.317654 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54dd90-d3eb-4744-b12e-be53f571cea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:13.317929 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54dd90-d3eb-4744-b12e-be53f571cea2 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:13.318582 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54dd90-d3eb-4744-b12e-be53f571cea2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:57:13.318884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54dd90-d3eb-4744-b12e-be53f571cea2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:13.320515 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54dd90-d3eb-4744-b12e-be53f571cea2 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:13.328113 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 434/875] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:57:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:57:13.334525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ca7d4-bf82-4415-b5cd-d8237aec94d0 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:13.379547 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:13.432107 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ca7d4-bf82-4415-b5cd-d8237aec94d0 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:13.467949 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682e36c-016f-4890-ab23-8baadb33912d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:13.468605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682e36c-016f-4890-ab23-8baadb33912d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:13.468890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682e36c-016f-4890-ab23-8baadb33912d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:13.469167 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682e36c-016f-4890-ab23-8baadb33912d None admin] PATH_INFO: `/v3/users/5655b57378a9439fb0b7d71ebed7de25` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:13.470280 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682e36c-016f-4890-ab23-8baadb33912d None admin] RBAC: Authorizing `identity:get_user(user_id=5655b57378a9439fb0b7d71ebed7de25)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:13.471745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682e36c-016f-4890-ab23-8baadb33912d None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:13.473321 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 442/876] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:57:13 2022] GET /identity/v3/users/5655b57378a9439fb0b7d71ebed7de25 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:57:13.479027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54dd90-d3eb-4744-b12e-be53f571cea2 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:13.520065 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:13.577519 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54dd90-d3eb-4744-b12e-be53f571cea2 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:13.609596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb453a05-6248-4488-a792-7535f115c71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:13.610189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb453a05-6248-4488-a792-7535f115c71f None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:13.610470 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb453a05-6248-4488-a792-7535f115c71f None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:13.610744 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb453a05-6248-4488-a792-7535f115c71f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:13.675121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb453a05-6248-4488-a792-7535f115c71f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:13.676574 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb453a05-6248-4488-a792-7535f115c71f None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:13.714063 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:13.771152 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 435/877] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:57:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 22 23:57:13.777042 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682e36c-016f-4890-ab23-8baadb33912d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:13.874620 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682e36c-016f-4890-ab23-8baadb33912d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:13.887365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:13.908611 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb7a27-1198-4d48-bf22-3ff23a57d162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:13.909226 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb7a27-1198-4d48-bf22-3ff23a57d162 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:13.909506 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb7a27-1198-4d48-bf22-3ff23a57d162 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:13.909779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb7a27-1198-4d48-bf22-3ff23a57d162 None admin] PATH_INFO: `/v3/projects/b50f052204cd4b22aeddbbaf73af76f6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:13.910877 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb7a27-1198-4d48-bf22-3ff23a57d162 None admin] RBAC: Authorizing `identity:get_project(project_id=b50f052204cd4b22aeddbbaf73af76f6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:13.912298 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb7a27-1198-4d48-bf22-3ff23a57d162 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:13.913943 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 443/878] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:57:13 2022] GET /identity/v3/projects/b50f052204cd4b22aeddbbaf73af76f6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:57:13.919970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb453a05-6248-4488-a792-7535f115c71f None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:14.016302 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb453a05-6248-4488-a792-7535f115c71f None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:14.049816 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f62b05-03dc-4bc3-be11-3b9926e50185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:14.050413 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f62b05-03dc-4bc3-be11-3b9926e50185 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:14.050696 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f62b05-03dc-4bc3-be11-3b9926e50185 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:14.050975 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f62b05-03dc-4bc3-be11-3b9926e50185 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:14.051682 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f62b05-03dc-4bc3-be11-3b9926e50185 None admin] RBAC: Adding query filter params (role.id=839a55263e2c4cbaa2f945b75cf298d6, scope.project.id=b50f052204cd4b22aeddbbaf73af76f6, user.id=5655b57378a9439fb0b7d71ebed7de25) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:57:14.051986 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f62b05-03dc-4bc3-be11-3b9926e50185 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:14.053427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f62b05-03dc-4bc3-be11-3b9926e50185 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:14.058919 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 436/879] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:57:13 2022] GET /identity/v3/role_assignments?user.id=5655b57378a9439fb0b7d71ebed7de25&scope.project.id=b50f052204cd4b22aeddbbaf73af76f6&role.id=839a55263e2c4cbaa2f945b75cf298d6 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:57:16.204019 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350d6fc-bec7-4f6b-bae8-f62c8f4c4492 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:16.204666 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350d6fc-bec7-4f6b-bae8-f62c8f4c4492 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:16.204666 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350d6fc-bec7-4f6b-bae8-f62c8f4c4492 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:16.206470 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 444/880] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:57:16.211809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55384176-22bc-4d4a-8bd3-eb3f75739f3d None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:16.212249 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55384176-22bc-4d4a-8bd3-eb3f75739f3d None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:16.212347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55384176-22bc-4d4a-8bd3-eb3f75739f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:16.219510 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:16.231551 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-55384176-22bc-4d4a-8bd3-eb3f75739f3d None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:57:16.232771 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-55384176-22bc-4d4a-8bd3-eb3f75739f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:57:16.297064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 437/881] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:57:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:57:16.572986 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350d6fc-bec7-4f6b-bae8-f62c8f4c4492 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:16.671948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350d6fc-bec7-4f6b-bae8-f62c8f4c4492 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:16.704373 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6a2da-2ad5-4d5d-9632-e17cceb53793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:16.705544 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6a2da-2ad5-4d5d-9632-e17cceb53793 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:16.705833 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6a2da-2ad5-4d5d-9632-e17cceb53793 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:16.706110 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6a2da-2ad5-4d5d-9632-e17cceb53793 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:16.706767 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6a2da-2ad5-4d5d-9632-e17cceb53793 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:16.708098 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6a2da-2ad5-4d5d-9632-e17cceb53793 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:16.723400 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 445/882] 10.222.0.13 () {60 vars in 1138 bytes} [Sat Oct 22 23:57:16 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:57:18.888739 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096af5e-3874-4ca2-9464-cfd3441e05e3 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:18.889310 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096af5e-3874-4ca2-9464-cfd3441e05e3 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:18.889310 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096af5e-3874-4ca2-9464-cfd3441e05e3 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:18.890645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 438/883] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:57:18.896458 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bef01a-2fc5-42d1-bc3b-b276acd54063 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:18.896864 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bef01a-2fc5-42d1-bc3b-b276acd54063 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:18.897054 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bef01a-2fc5-42d1-bc3b-b276acd54063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:18.914962 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-91bef01a-2fc5-42d1-bc3b-b276acd54063 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:57:18.916150 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-91bef01a-2fc5-42d1-bc3b-b276acd54063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:57:18.949887 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:18.983113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 446/884] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:57:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:57:19.252139 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096af5e-3874-4ca2-9464-cfd3441e05e3 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:19.351419 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096af5e-3874-4ca2-9464-cfd3441e05e3 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:19.383709 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cca82d-97c8-4d14-a092-2e3422b2cba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:19.384316 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cca82d-97c8-4d14-a092-2e3422b2cba3 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:19.384638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cca82d-97c8-4d14-a092-2e3422b2cba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:19.384916 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cca82d-97c8-4d14-a092-2e3422b2cba3 None admin] PATH_INFO: `/v3/domains/dcb12d4b02c148cfb12e03d37b19de30` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:19.386733 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:19.396416 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cca82d-97c8-4d14-a092-2e3422b2cba3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=dcb12d4b02c148cfb12e03d37b19de30)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:19.397862 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cca82d-97c8-4d14-a092-2e3422b2cba3 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:19.399430 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 439/885] 10.222.0.13 () {56 vars in 1183 bytes} [Sat Oct 22 23:57:19 2022] GET /identity/v3/domains/dcb12d4b02c148cfb12e03d37b19de30 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:57:19.405340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bef01a-2fc5-42d1-bc3b-b276acd54063 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:19.501082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bef01a-2fc5-42d1-bc3b-b276acd54063 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:19.534006 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d4dbe-9976-4701-8e11-48a1057d77d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:19.534545 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d4dbe-9976-4701-8e11-48a1057d77d0 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:19.534830 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d4dbe-9976-4701-8e11-48a1057d77d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:19.535106 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d4dbe-9976-4701-8e11-48a1057d77d0 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:19.535743 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d4dbe-9976-4701-8e11-48a1057d77d0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:19.537186 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d4dbe-9976-4701-8e11-48a1057d77d0 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:19.562857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 447/886] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Oct 22 23:57:19 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:57:21.737479 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3cc02-8154-4a74-98e4-7f01d3dc59b1 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:21.738012 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3cc02-8154-4a74-98e4-7f01d3dc59b1 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:21.738012 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3cc02-8154-4a74-98e4-7f01d3dc59b1 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:21.739389 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 440/887] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:57:21.744751 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eaeb89-81b8-41e7-8ef0-a8c130d09644 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:21.745176 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eaeb89-81b8-41e7-8ef0-a8c130d09644 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:21.745313 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eaeb89-81b8-41e7-8ef0-a8c130d09644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:21.763621 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-49eaeb89-81b8-41e7-8ef0-a8c130d09644 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:57:21.764845 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-49eaeb89-81b8-41e7-8ef0-a8c130d09644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:57:21.781078 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:21.831266 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 448/888] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:57:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:57:22.098504 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3cc02-8154-4a74-98e4-7f01d3dc59b1 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:22.198231 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3cc02-8154-4a74-98e4-7f01d3dc59b1 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:22.231538 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334a8e7-22b8-447c-bd3f-2acaf7f1a355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:22.232147 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334a8e7-22b8-447c-bd3f-2acaf7f1a355 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:22.232425 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334a8e7-22b8-447c-bd3f-2acaf7f1a355 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:22.232722 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334a8e7-22b8-447c-bd3f-2acaf7f1a355 None admin] PATH_INFO: `/v3/domains/dcb12d4b02c148cfb12e03d37b19de30` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:22.233814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0334a8e7-22b8-447c-bd3f-2acaf7f1a355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=dcb12d4b02c148cfb12e03d37b19de30)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:22.235224 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0334a8e7-22b8-447c-bd3f-2acaf7f1a355 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:22.237308 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 441/889] 10.222.0.13 () {56 vars in 1183 bytes} [Sat Oct 22 23:57:22 2022] GET /identity/v3/domains/dcb12d4b02c148cfb12e03d37b19de30 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:57:22.243089 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eaeb89-81b8-41e7-8ef0-a8c130d09644 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:22.337308 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eaeb89-81b8-41e7-8ef0-a8c130d09644 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:22.369544 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8bb60d-8a9e-4564-93e3-daa48ef5ad68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:22.370075 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bb60d-8a9e-4564-93e3-daa48ef5ad68 None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:22.370352 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bb60d-8a9e-4564-93e3-daa48ef5ad68 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:22.370623 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bb60d-8a9e-4564-93e3-daa48ef5ad68 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:22.371253 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8bb60d-8a9e-4564-93e3-daa48ef5ad68 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:22.372685 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8bb60d-8a9e-4564-93e3-daa48ef5ad68 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:22.391714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 449/890] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Oct 22 23:57:22 2022] POST /identity/v3/users => generated 311 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:57:24.698678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8c446-f1ae-4b7d-8fa7-1b7a6771e5ee None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:24.699232 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8c446-f1ae-4b7d-8fa7-1b7a6771e5ee None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:24.699232 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8c446-f1ae-4b7d-8fa7-1b7a6771e5ee None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:24.700660 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 442/891] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:57:24.706349 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180f45d-61ff-4ec5-9815-4048a20c7c1b None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:24.706733 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180f45d-61ff-4ec5-9815-4048a20c7c1b None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:24.706960 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180f45d-61ff-4ec5-9815-4048a20c7c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:24.725561 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-6180f45d-61ff-4ec5-9815-4048a20c7c1b None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:57:24.726753 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-6180f45d-61ff-4ec5-9815-4048a20c7c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:57:24.728726 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:24.793505 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 450/892] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:57:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:57:25.065203 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8c446-f1ae-4b7d-8fa7-1b7a6771e5ee None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:25.128427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:25.167174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8c446-f1ae-4b7d-8fa7-1b7a6771e5ee None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:25.200869 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255d05f-11b6-464e-bae0-693eea238321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:25.201484 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255d05f-11b6-464e-bae0-693eea238321 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:25.201766 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255d05f-11b6-464e-bae0-693eea238321 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:25.202040 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255d05f-11b6-464e-bae0-693eea238321 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:25.211791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255d05f-11b6-464e-bae0-693eea238321 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:57:25.216532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-d255d05f-11b6-464e-bae0-693eea238321 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:57:25.216532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:57:25.216532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:57:25.216532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:57:25.216532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:57:25.216532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:57:25.216532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:57:25.216532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:57:25.216532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:57:25.216532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:57:25.216532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:57:25.216532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:57:25.216532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:57:25.216532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:57:25.216532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:57:25.216532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:57:25.217720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:57:25.217720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:57:25.217720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:57:25.217720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:57:25.217720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:57:25.217720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:57:25.217720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:57:25.217720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:57:25.217720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:57:25.217720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:57:25.217720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:57:25.217720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:57:25.217720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:57:25.217720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:57:25.217720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:57:25.217720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:57:25.218842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:57:25.218842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:57:25.218842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:57:25.218842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:57:25.218842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:57:25.218842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:57:25.218842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:57:25.218842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:57:25.218842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:57:25.218842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:57:25.218842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:57:25.218842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:57:25.218842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 443/893] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Oct 22 23:57:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:57:25.223741 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180f45d-61ff-4ec5-9815-4048a20c7c1b None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:25.319865 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180f45d-61ff-4ec5-9815-4048a20c7c1b None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:25.353160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036b0b4-b1c8-4dbb-a7c3-63f35a7aade5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:25.353758 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036b0b4-b1c8-4dbb-a7c3-63f35a7aade5 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:25.354045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036b0b4-b1c8-4dbb-a7c3-63f35a7aade5 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:25.354322 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036b0b4-b1c8-4dbb-a7c3-63f35a7aade5 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:25.354975 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036b0b4-b1c8-4dbb-a7c3-63f35a7aade5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:57:25.355281 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036b0b4-b1c8-4dbb-a7c3-63f35a7aade5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:25.356660 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036b0b4-b1c8-4dbb-a7c3-63f35a7aade5 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:25.362255 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:25.365644 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 451/894] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:57:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:57:25.371105 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255d05f-11b6-464e-bae0-693eea238321 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:25.466868 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255d05f-11b6-464e-bae0-693eea238321 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:25.488968 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:25.500701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4233db3-6584-404d-982d-c5924ccf6372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:25.501318 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4233db3-6584-404d-982d-c5924ccf6372 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:25.501603 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4233db3-6584-404d-982d-c5924ccf6372 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:25.501900 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4233db3-6584-404d-982d-c5924ccf6372 None admin] PATH_INFO: `/v3/users/c47ef0872238427c9175306384f61859` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:25.515839 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4233db3-6584-404d-982d-c5924ccf6372 None admin] RBAC: Authorizing `identity:get_user(user_id=c47ef0872238427c9175306384f61859)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:25.517294 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4233db3-6584-404d-982d-c5924ccf6372 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:25.519104 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 444/895] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:57:25 2022] GET /identity/v3/users/c47ef0872238427c9175306384f61859 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:57:25.524840 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036b0b4-b1c8-4dbb-a7c3-63f35a7aade5 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:25.622193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036b0b4-b1c8-4dbb-a7c3-63f35a7aade5 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:25.654505 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f92eb2-4abf-44d6-b4d5-a26beae26ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:25.655115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f92eb2-4abf-44d6-b4d5-a26beae26ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:25.655395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f92eb2-4abf-44d6-b4d5-a26beae26ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:25.655666 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f92eb2-4abf-44d6-b4d5-a26beae26ae4 None admin] PATH_INFO: `/v3/projects/3cfbd8b577934b3892ab9abbdb17c2cb` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:25.656786 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f92eb2-4abf-44d6-b4d5-a26beae26ae4 None admin] RBAC: Authorizing `identity:get_project(project_id=3cfbd8b577934b3892ab9abbdb17c2cb)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:25.658714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f92eb2-4abf-44d6-b4d5-a26beae26ae4 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:25.660824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 452/896] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:57:25 2022] GET /identity/v3/projects/3cfbd8b577934b3892ab9abbdb17c2cb => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:57:25.666565 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4233db3-6584-404d-982d-c5924ccf6372 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:25.763355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4233db3-6584-404d-982d-c5924ccf6372 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:25.796289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b58b79-19ef-4add-b209-a67a7551ece3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:25.796953 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b58b79-19ef-4add-b209-a67a7551ece3 None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:25.797234 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b58b79-19ef-4add-b209-a67a7551ece3 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:25.797505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b58b79-19ef-4add-b209-a67a7551ece3 None admin] PATH_INFO: `/v3/projects/3cfbd8b577934b3892ab9abbdb17c2cb/users/c47ef0872238427c9175306384f61859/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:25.800373 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b58b79-19ef-4add-b209-a67a7551ece3 None admin] RBAC: Authorizing `identity:create_grant(project_id=3cfbd8b577934b3892ab9abbdb17c2cb, user_id=c47ef0872238427c9175306384f61859, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:25.801801 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b58b79-19ef-4add-b209-a67a7551ece3 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:25.813924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 445/897] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:57:25 2022] PUT /identity/v3/projects/3cfbd8b577934b3892ab9abbdb17c2cb/users/c47ef0872238427c9175306384f61859/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:57:27.947136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3009f1-cbf7-4a41-b365-3f88211b9395 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:27.947687 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3009f1-cbf7-4a41-b365-3f88211b9395 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:27.947687 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3009f1-cbf7-4a41-b365-3f88211b9395 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:27.949059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 453/898] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:57:27.954115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f89da-d797-429c-b525-a496ba3e6f2b None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:27.954531 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f89da-d797-429c-b525-a496ba3e6f2b None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:27.954663 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f89da-d797-429c-b525-a496ba3e6f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:27.973659 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-8b1f89da-d797-429c-b525-a496ba3e6f2b None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:57:27.974828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8b1f89da-d797-429c-b525-a496ba3e6f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:57:28.012980 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:28.045951 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 446/899] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:57:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:57:28.314347 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3009f1-cbf7-4a41-b365-3f88211b9395 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:28.385650 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:28.419053 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3009f1-cbf7-4a41-b365-3f88211b9395 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:28.453629 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e59717-39ea-4aae-ae52-9c02e8ff109a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:28.454239 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e59717-39ea-4aae-ae52-9c02e8ff109a None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:28.454514 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e59717-39ea-4aae-ae52-9c02e8ff109a None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:28.454782 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e59717-39ea-4aae-ae52-9c02e8ff109a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:28.464920 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e59717-39ea-4aae-ae52-9c02e8ff109a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:57:28.469675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-c2e59717-39ea-4aae-ae52-9c02e8ff109a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:57:28.469675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:57:28.469675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:57:28.469675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:57:28.469675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:57:28.469675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:57:28.469675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:57:28.469675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:57:28.469675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:57:28.469675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:57:28.469675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:57:28.469675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:57:28.469675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:57:28.469675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:57:28.469675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:57:28.469675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:57:28.470796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:57:28.470796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:57:28.470796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:57:28.470796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:57:28.470796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:57:28.470796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:57:28.470796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:57:28.470796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:57:28.470796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:57:28.470796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:57:28.470796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:57:28.470796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:57:28.470796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:57:28.470796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:57:28.470796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:57:28.470796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:57:28.471713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:57:28.471713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:57:28.471713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:57:28.471713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:57:28.471713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:57:28.471713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:57:28.471713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:57:28.471713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:57:28.471713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:57:28.471713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:57:28.471713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:57:28.471713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:57:28.471713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 454/900] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Oct 22 23:57:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:57:28.476867 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f89da-d797-429c-b525-a496ba3e6f2b None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:28.573031 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f89da-d797-429c-b525-a496ba3e6f2b None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:28.605416 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af79a6-11e7-489d-9759-a0b60376cc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:28.606004 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af79a6-11e7-489d-9759-a0b60376cc16 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:28.606282 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af79a6-11e7-489d-9759-a0b60376cc16 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:28.606552 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af79a6-11e7-489d-9759-a0b60376cc16 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:28.607197 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af79a6-11e7-489d-9759-a0b60376cc16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:57:28.607497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af79a6-11e7-489d-9759-a0b60376cc16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:28.608867 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af79a6-11e7-489d-9759-a0b60376cc16 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:28.617412 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 447/901] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:57:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:57:28.623426 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e59717-39ea-4aae-ae52-9c02e8ff109a None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:28.726550 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e59717-39ea-4aae-ae52-9c02e8ff109a None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:28.752709 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:28.760816 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0bb9dc-3501-4f8d-bb2b-64f3ad3e3056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:28.761436 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bb9dc-3501-4f8d-bb2b-64f3ad3e3056 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:28.761720 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bb9dc-3501-4f8d-bb2b-64f3ad3e3056 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:28.761996 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0bb9dc-3501-4f8d-bb2b-64f3ad3e3056 None admin] PATH_INFO: `/v3/users/c47ef0872238427c9175306384f61859` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:28.763118 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0bb9dc-3501-4f8d-bb2b-64f3ad3e3056 None admin] RBAC: Authorizing `identity:get_user(user_id=c47ef0872238427c9175306384f61859)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:28.764552 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0bb9dc-3501-4f8d-bb2b-64f3ad3e3056 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:28.766127 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 455/902] 10.222.0.13 () {56 vars in 1177 bytes} [Sat Oct 22 23:57:28 2022] GET /identity/v3/users/c47ef0872238427c9175306384f61859 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:57:28.771980 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af79a6-11e7-489d-9759-a0b60376cc16 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:28.803578 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:28.870804 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af79a6-11e7-489d-9759-a0b60376cc16 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:28.903316 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a46e3-b639-46d9-a868-dd0b76e6080b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:28.903900 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a46e3-b639-46d9-a868-dd0b76e6080b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:28.904178 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a46e3-b639-46d9-a868-dd0b76e6080b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:28.904448 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a46e3-b639-46d9-a868-dd0b76e6080b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:28.969680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a46e3-b639-46d9-a868-dd0b76e6080b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:28.971482 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a46e3-b639-46d9-a868-dd0b76e6080b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:28.999816 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:29.067116 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 448/903] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:57:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 22 23:57:29.073206 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0bb9dc-3501-4f8d-bb2b-64f3ad3e3056 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:29.170919 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0bb9dc-3501-4f8d-bb2b-64f3ad3e3056 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:29.204836 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f135b2b-25f0-4c4b-aee3-f64927ec337c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:29.205505 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f135b2b-25f0-4c4b-aee3-f64927ec337c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:29.205788 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f135b2b-25f0-4c4b-aee3-f64927ec337c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:29.206064 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f135b2b-25f0-4c4b-aee3-f64927ec337c None admin] PATH_INFO: `/v3/projects/3cfbd8b577934b3892ab9abbdb17c2cb` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:29.207173 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f135b2b-25f0-4c4b-aee3-f64927ec337c None admin] RBAC: Authorizing `identity:get_project(project_id=3cfbd8b577934b3892ab9abbdb17c2cb)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:29.208650 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f135b2b-25f0-4c4b-aee3-f64927ec337c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:29.210268 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 456/904] 10.222.0.13 () {56 vars in 1186 bytes} [Sat Oct 22 23:57:29 2022] GET /identity/v3/projects/3cfbd8b577934b3892ab9abbdb17c2cb => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:57:29.216421 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a46e3-b639-46d9-a868-dd0b76e6080b None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:57:29.312268 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a46e3-b639-46d9-a868-dd0b76e6080b None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:57:29.340771 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:29.346581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de2ac7-d0a9-422a-9979-ef5b82d28d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:57:29.347172 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de2ac7-d0a9-422a-9979-ef5b82d28d33 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:57:29.347453 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de2ac7-d0a9-422a-9979-ef5b82d28d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:57:29.347719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de2ac7-d0a9-422a-9979-ef5b82d28d33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:57:29.348421 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de2ac7-d0a9-422a-9979-ef5b82d28d33 None admin] RBAC: Adding query filter params (role.id=839a55263e2c4cbaa2f945b75cf298d6, scope.project.id=3cfbd8b577934b3892ab9abbdb17c2cb, user.id=c47ef0872238427c9175306384f61859) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:57:29.348756 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de2ac7-d0a9-422a-9979-ef5b82d28d33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:57:29.350163 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de2ac7-d0a9-422a-9979-ef5b82d28d33 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:57:29.355776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 449/905] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:57:29 2022] GET /identity/v3/role_assignments?user.id=c47ef0872238427c9175306384f61859&scope.project.id=3cfbd8b577934b3892ab9abbdb17c2cb&role.id=839a55263e2c4cbaa2f945b75cf298d6 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:57:39.200837 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=68 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:57:39.353204 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:24.849543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7bad7-f59b-494e-a0f0-0647f8407337 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:24.850155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7bad7-f59b-494e-a0f0-0647f8407337 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:24.850155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7bad7-f59b-494e-a0f0-0647f8407337 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:24.851637 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 457/906] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:58:24.858640 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c86f9-f0e1-4e86-a357-87b9cd944f8d None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:24.858640 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c86f9-f0e1-4e86-a357-87b9cd944f8d None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:24.859289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c86f9-f0e1-4e86-a357-87b9cd944f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:24.886783 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-0f3c86f9-f0e1-4e86-a357-87b9cd944f8d None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:58:24.888537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-0f3c86f9-f0e1-4e86-a357-87b9cd944f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:58:24.974635 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 450/907] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:58:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:58:25.258762 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7bad7-f59b-494e-a0f0-0647f8407337 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:25.373696 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7bad7-f59b-494e-a0f0-0647f8407337 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:25.414804 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cef59b-1b71-455b-b069-cb92c4febfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:25.415436 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cef59b-1b71-455b-b069-cb92c4febfa7 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:25.415721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cef59b-1b71-455b-b069-cb92c4febfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:25.416002 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cef59b-1b71-455b-b069-cb92c4febfa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:25.427317 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cef59b-1b71-455b-b069-cb92c4febfa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:25.428907 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cef59b-1b71-455b-b069-cb92c4febfa7 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:25.449213 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-f7cef59b-1b71-455b-b069-cb92c4febfa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:58:25.449213 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:25.449213 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:58:25.449213 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:58:25.449213 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:58:25.449213 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:58:25.449213 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:58:25.449213 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:58:25.449213 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:58:25.449213 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:58:25.449213 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:58:25.449213 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:58:25.449213 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:58:25.449213 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:58:25.449213 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:58:25.449213 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:58:25.450292 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:58:25.450292 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:58:25.450292 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:58:25.450292 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:58:25.450292 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:58:25.450292 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:58:25.450292 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:58:25.450292 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:58:25.450292 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:58:25.450292 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:58:25.450292 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:58:25.450292 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:58:25.450292 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:58:25.450292 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:58:25.450292 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:58:25.450292 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:58:25.450292 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:58:25.451444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:58:25.451444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:58:25.451444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:58:25.451444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 458/908] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:58:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:58:25.459387 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c86f9-f0e1-4e86-a357-87b9cd944f8d None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:25.591557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c86f9-f0e1-4e86-a357-87b9cd944f8d None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:25.624867 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:25.632280 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6148178-e473-4b86-abb9-ef7e7a7198dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:25.633037 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6148178-e473-4b86-abb9-ef7e7a7198dd None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:25.633378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6148178-e473-4b86-abb9-ef7e7a7198dd None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:25.633659 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6148178-e473-4b86-abb9-ef7e7a7198dd None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:25.634309 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6148178-e473-4b86-abb9-ef7e7a7198dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:25.634614 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6148178-e473-4b86-abb9-ef7e7a7198dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:25.635982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6148178-e473-4b86-abb9-ef7e7a7198dd None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:25.653034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 451/909] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:58:25.660349 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cef59b-1b71-455b-b069-cb92c4febfa7 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:25.749397 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:25.823038 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cef59b-1b71-455b-b069-cb92c4febfa7 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:25.880051 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e59a0-ad8a-4b05-968a-817fd5d355cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:25.881946 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e59a0-ad8a-4b05-968a-817fd5d355cf None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:25.882603 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e59a0-ad8a-4b05-968a-817fd5d355cf None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:25.883375 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e59a0-ad8a-4b05-968a-817fd5d355cf None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:25.884261 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e59a0-ad8a-4b05-968a-817fd5d355cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:25.885923 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e59a0-ad8a-4b05-968a-817fd5d355cf None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:25.918318 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 459/910] 10.222.0.13 () {60 vars in 1132 bytes} [Sat Oct 22 23:58:25 2022] POST /identity/v3/users => generated 251 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:58:28.536755 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b4079-ace4-4d99-9969-0d6ba19a288e None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:28.537285 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b4079-ace4-4d99-9969-0d6ba19a288e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:28.537285 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b4079-ace4-4d99-9969-0d6ba19a288e None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:28.538563 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 452/911] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:58:28.543956 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf2c1e-009b-4411-b226-73a7691b27cc None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:28.544406 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf2c1e-009b-4411-b226-73a7691b27cc None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:28.544554 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf2c1e-009b-4411-b226-73a7691b27cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:28.568113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-d9bf2c1e-009b-4411-b226-73a7691b27cc None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:58:28.569337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d9bf2c1e-009b-4411-b226-73a7691b27cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:58:28.650507 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 460/912] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:58:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:58:28.953390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b4079-ace4-4d99-9969-0d6ba19a288e None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:29.079706 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b4079-ace4-4d99-9969-0d6ba19a288e None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:29.125203 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258650f-5e54-4987-9534-730922c1dabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:29.125839 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258650f-5e54-4987-9534-730922c1dabc None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:29.126122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258650f-5e54-4987-9534-730922c1dabc None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:29.126397 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258650f-5e54-4987-9534-730922c1dabc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:29.141577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e258650f-5e54-4987-9534-730922c1dabc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:58:29.147026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-e258650f-5e54-4987-9534-730922c1dabc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:58:29.147026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:29.147026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:58:29.147026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:58:29.147026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:58:29.147026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:58:29.147026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:58:29.147026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:58:29.147026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:58:29.147026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:58:29.147026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:58:29.147026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:58:29.147026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:58:29.147026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:58:29.147026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:58:29.147026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:58:29.152648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:58:29.152648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:58:29.152648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:58:29.152648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:58:29.152648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:58:29.152648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:58:29.152648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:58:29.152648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:58:29.152648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:58:29.152648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:58:29.152648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:58:29.152648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:58:29.152648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:58:29.152648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:58:29.152648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:58:29.152648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:58:29.153717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:58:29.153717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:58:29.153717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:58:29.153717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:58:29.153717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:58:29.153717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:58:29.153717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:58:29.153717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:58:29.153717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:58:29.153717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:58:29.153717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:58:29.153717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:58:29.153717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 453/913] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Oct 22 23:58:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:58:29.154742 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf2c1e-009b-4411-b226-73a7691b27cc None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:29.201667 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:29.271608 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf2c1e-009b-4411-b226-73a7691b27cc None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:29.317115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf829a-f835-4da1-8943-daebc9df4d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:29.318281 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf829a-f835-4da1-8943-daebc9df4d96 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:29.318281 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf829a-f835-4da1-8943-daebc9df4d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:29.318281 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf829a-f835-4da1-8943-daebc9df4d96 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:29.318972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf829a-f835-4da1-8943-daebc9df4d96 None admin] RBAC: Adding query filter params (name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:29.319278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf829a-f835-4da1-8943-daebc9df4d96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:29.320777 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf829a-f835-4da1-8943-daebc9df4d96 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:29.335133 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 461/914] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Oct 22 23:58:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:58:29.335329 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258650f-5e54-4987-9534-730922c1dabc None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:29.395270 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:29.449461 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258650f-5e54-4987-9534-730922c1dabc None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:29.488234 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898d77c-e4d1-443a-9556-d3b9b34eadf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:29.489049 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898d77c-e4d1-443a-9556-d3b9b34eadf8 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:29.489451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898d77c-e4d1-443a-9556-d3b9b34eadf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:29.489824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898d77c-e4d1-443a-9556-d3b9b34eadf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:29.500230 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e898d77c-e4d1-443a-9556-d3b9b34eadf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:29.501893 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e898d77c-e4d1-443a-9556-d3b9b34eadf8 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:29.518916 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-e898d77c-e4d1-443a-9556-d3b9b34eadf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:58:29.518916 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:29.518916 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:58:29.518916 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:58:29.518916 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:58:29.518916 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:58:29.518916 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:58:29.518916 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:58:29.518916 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:58:29.518916 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:58:29.518916 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:58:29.518916 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:58:29.518916 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:58:29.518916 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:58:29.518916 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:58:29.518916 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:58:29.520454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:58:29.520454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:58:29.520454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:58:29.520454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:58:29.520454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:58:29.520454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:58:29.520454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:58:29.520454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:58:29.520454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:58:29.520454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:58:29.520454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:58:29.520454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:58:29.520454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:58:29.520454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:58:29.520454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:58:29.520454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:58:29.520454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:58:29.521912 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:58:29.521912 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:58:29.521912 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:58:29.522731 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 454/915] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:58:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:58:29.529152 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf829a-f835-4da1-8943-daebc9df4d96 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:29.662976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:29.672524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf829a-f835-4da1-8943-daebc9df4d96 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:29.707579 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf309bd-08e6-42f2-b84a-461069697f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:29.708785 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf309bd-08e6-42f2-b84a-461069697f54 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:29.709256 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf309bd-08e6-42f2-b84a-461069697f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:29.709717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf309bd-08e6-42f2-b84a-461069697f54 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:29.710771 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf309bd-08e6-42f2-b84a-461069697f54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:29.711163 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf309bd-08e6-42f2-b84a-461069697f54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:29.713344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf309bd-08e6-42f2-b84a-461069697f54 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:29.738922 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898d77c-e4d1-443a-9556-d3b9b34eadf8 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:29.739471 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 462/916] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:58:29.851354 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898d77c-e4d1-443a-9556-d3b9b34eadf8 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:29.853187 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:29.890963 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836d93f-cef7-456e-ac48-8cdb8fdc0a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:29.891875 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836d93f-cef7-456e-ac48-8cdb8fdc0a1b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:29.892318 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836d93f-cef7-456e-ac48-8cdb8fdc0a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:29.892786 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836d93f-cef7-456e-ac48-8cdb8fdc0a1b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:29.900872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4836d93f-cef7-456e-ac48-8cdb8fdc0a1b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:29.902318 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4836d93f-cef7-456e-ac48-8cdb8fdc0a1b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:29.913846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-4836d93f-cef7-456e-ac48-8cdb8fdc0a1b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 22 23:58:29.913846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:29.913846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:58:29.913846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:58:29.913846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:58:29.913846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:58:29.913846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:58:29.913846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:58:29.913846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:58:29.913846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:58:29.913846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:58:29.913846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:58:29.913846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:58:29.913846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:58:29.913846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:58:29.913846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:58:29.914851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:58:29.914851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:58:29.914851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:58:29.914851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:58:29.914851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:58:29.914851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:58:29.914851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:58:29.914851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:58:29.914851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:58:29.914851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:58:29.914851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:58:29.914851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:58:29.914851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:58:29.914851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:58:29.914851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:58:29.914851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:58:29.914851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:58:29.914851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:58:29.922681 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:58:29.922681 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:58:29.922681 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:58:29.922681 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:58:29.922681 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:58:29.922681 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:58:29.922681 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:58:29.922681 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:58:29.922681 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 22 23:58:29.922681 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:58:29.922681 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 455/917] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:58:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:58:29.923443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf309bd-08e6-42f2-b84a-461069697f54 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:30.042865 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf309bd-08e6-42f2-b84a-461069697f54 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:30.083112 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e2779-5f95-4d83-8961-a7b2e908cc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:30.083835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e2779-5f95-4d83-8961-a7b2e908cc93 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:30.084192 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e2779-5f95-4d83-8961-a7b2e908cc93 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:30.084575 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e2779-5f95-4d83-8961-a7b2e908cc93 None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:30.085313 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e2779-5f95-4d83-8961-a7b2e908cc93 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:30.085689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e2779-5f95-4d83-8961-a7b2e908cc93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:30.087035 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e2779-5f95-4d83-8961-a7b2e908cc93 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:30.112899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 463/918] 10.222.0.13 () {56 vars in 1143 bytes} [Sat Oct 22 23:58:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:58:30.118882 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836d93f-cef7-456e-ac48-8cdb8fdc0a1b None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:30.237969 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836d93f-cef7-456e-ac48-8cdb8fdc0a1b None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:30.272835 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33d69a-57d5-4c5a-86be-3c2614615773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:30.273487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33d69a-57d5-4c5a-86be-3c2614615773 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:30.273753 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33d69a-57d5-4c5a-86be-3c2614615773 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:30.274006 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b33d69a-57d5-4c5a-86be-3c2614615773 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:30.285795 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33d69a-57d5-4c5a-86be-3c2614615773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:30.287240 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b33d69a-57d5-4c5a-86be-3c2614615773 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:30.302338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-3b33d69a-57d5-4c5a-86be-3c2614615773 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:58:30.302338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:30.302338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:58:30.302338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:58:30.302338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:58:30.302338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:58:30.302338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:58:30.302338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:58:30.302338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:58:30.302338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:58:30.302338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:58:30.302338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:58:30.302338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:58:30.302338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:58:30.302338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:58:30.302338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:58:30.308254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:58:30.308254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:58:30.308254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:58:30.308254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:58:30.308254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:58:30.308254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:58:30.308254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:58:30.308254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:58:30.308254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:58:30.308254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:58:30.308254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:58:30.308254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:58:30.308254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:58:30.308254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:58:30.308254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:58:30.308254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:58:30.308254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:58:30.309255 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:58:30.309255 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:58:30.309255 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:58:30.309255 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 456/919] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:58:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:58:30.309561 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e2779-5f95-4d83-8961-a7b2e908cc93 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:30.329337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:30.433510 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e2779-5f95-4d83-8961-a7b2e908cc93 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:30.476057 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a22dc6-b187-445e-981f-e201928e672b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:30.476703 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a22dc6-b187-445e-981f-e201928e672b None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:30.477057 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a22dc6-b187-445e-981f-e201928e672b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:30.477301 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a22dc6-b187-445e-981f-e201928e672b None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:30.477887 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a22dc6-b187-445e-981f-e201928e672b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:30.478194 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a22dc6-b187-445e-981f-e201928e672b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:30.479673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a22dc6-b187-445e-981f-e201928e672b None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:30.492838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 464/920] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:58:30.498584 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33d69a-57d5-4c5a-86be-3c2614615773 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:30.515537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:30.621386 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b33d69a-57d5-4c5a-86be-3c2614615773 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:30.665185 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625efca-8373-4add-b860-c25f46d33479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:30.665941 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625efca-8373-4add-b860-c25f46d33479 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:30.666281 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625efca-8373-4add-b860-c25f46d33479 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:30.666682 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625efca-8373-4add-b860-c25f46d33479 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:30.675655 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625efca-8373-4add-b860-c25f46d33479 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:30.677551 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625efca-8373-4add-b860-c25f46d33479 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:30.692586 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-f625efca-8373-4add-b860-c25f46d33479 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:58:30.692586 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:30.692586 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:58:30.692586 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:58:30.692586 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:58:30.692586 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:58:30.692586 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:58:30.692586 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:58:30.692586 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:58:30.692586 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:58:30.692586 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:58:30.692586 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:58:30.692586 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:58:30.692586 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:58:30.692586 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:58:30.694337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:58:30.694337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:58:30.694337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:58:30.694337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:58:30.694337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:58:30.694337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:58:30.694337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:58:30.694337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:58:30.694337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:58:30.694337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:58:30.694337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:58:30.694337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:58:30.694337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:58:30.694337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:58:30.694337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:58:30.694337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:58:30.694337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:58:30.695541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:58:30.695541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:58:30.695541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:58:30.695541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:58:30.695541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:58:30.695541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:58:30.695541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:58:30.695541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:58:30.700578 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 457/921] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:58:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:58:30.701706 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a22dc6-b187-445e-981f-e201928e672b None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:30.795572 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:30.818793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a22dc6-b187-445e-981f-e201928e672b None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:30.861723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38345a7-de9e-4bc7-af75-f349f5fcd361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:30.862489 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38345a7-de9e-4bc7-af75-f349f5fcd361 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:30.862809 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38345a7-de9e-4bc7-af75-f349f5fcd361 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:30.863093 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38345a7-de9e-4bc7-af75-f349f5fcd361 None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:30.863977 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38345a7-de9e-4bc7-af75-f349f5fcd361 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:30.864396 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38345a7-de9e-4bc7-af75-f349f5fcd361 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:30.866164 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38345a7-de9e-4bc7-af75-f349f5fcd361 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:30.881813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 465/922] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Oct 22 23:58:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:58:30.889181 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625efca-8373-4add-b860-c25f46d33479 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:30.991727 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:31.024740 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625efca-8373-4add-b860-c25f46d33479 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:31.068716 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa7a96-1461-40f0-8c9b-47bf552581e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:31.069436 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa7a96-1461-40f0-8c9b-47bf552581e9 None admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:31.069799 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa7a96-1461-40f0-8c9b-47bf552581e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:31.070102 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa7a96-1461-40f0-8c9b-47bf552581e9 None admin] PATH_INFO: `/v3/projects/5c9246c059754504afee4711d113a3fe/users/24108b6355414f64ba5e7bf84011eee6/roles/73721a054c3842bba868ab52c1a3f8e0` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:31.088220 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa7a96-1461-40f0-8c9b-47bf552581e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c9246c059754504afee4711d113a3fe, user_id=24108b6355414f64ba5e7bf84011eee6, role_id=73721a054c3842bba868ab52c1a3f8e0)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:31.089965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fa7a96-1461-40f0-8c9b-47bf552581e9 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:31.103948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 458/923] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:58:30 2022] PUT /identity/v3/projects/5c9246c059754504afee4711d113a3fe/users/24108b6355414f64ba5e7bf84011eee6/roles/73721a054c3842bba868ab52c1a3f8e0 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:58:33.392686 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3166e2-05ff-4df8-b34e-7ed0f56f681d None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:33.393232 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3166e2-05ff-4df8-b34e-7ed0f56f681d None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:33.393232 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3166e2-05ff-4df8-b34e-7ed0f56f681d None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:33.394503 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 466/924] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:58:33.400065 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc038e9-5c29-4cb1-829f-e16e023e17a9 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:33.400546 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc038e9-5c29-4cb1-829f-e16e023e17a9 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:33.400654 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc038e9-5c29-4cb1-829f-e16e023e17a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:33.419253 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-2fc038e9-5c29-4cb1-829f-e16e023e17a9 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:58:33.420455 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2fc038e9-5c29-4cb1-829f-e16e023e17a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:58:33.492692 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 459/925] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:58:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:58:33.763096 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3166e2-05ff-4df8-b34e-7ed0f56f681d None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:33.873982 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3166e2-05ff-4df8-b34e-7ed0f56f681d None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:33.911969 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79653f5b-7aa9-44f9-84b5-891913cd0bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:33.912643 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79653f5b-7aa9-44f9-84b5-891913cd0bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:33.912922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79653f5b-7aa9-44f9-84b5-891913cd0bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:33.913195 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79653f5b-7aa9-44f9-84b5-891913cd0bb7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:33.920175 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:33.924698 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79653f5b-7aa9-44f9-84b5-891913cd0bb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:58:33.931606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-79653f5b-7aa9-44f9-84b5-891913cd0bb7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:58:33.931606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:33.931606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:58:33.931606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:58:33.931606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:58:33.931606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:58:33.931606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:58:33.931606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:58:33.931606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:58:33.931606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:58:33.931606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:58:33.931606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:58:33.931606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:58:33.931606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:58:33.931606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:58:33.931606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:58:33.932744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:58:33.932744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:58:33.932744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:58:33.932744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:58:33.932744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:58:33.932744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:58:33.932744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:58:33.932744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:58:33.932744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:58:33.932744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:58:33.932744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:58:33.932744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:58:33.932744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:58:33.932744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:58:33.932744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:58:33.932744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:58:33.933780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:58:33.933780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:58:33.933780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:58:33.933780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:58:33.933780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:58:33.933780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:58:33.933780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:58:33.933780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:58:33.933780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:58:33.933780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:58:33.933780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 22 23:58:33.933780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:58:33.933780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 467/926] 10.222.0.13 () {56 vars in 1102 bytes} [Sat Oct 22 23:58:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:58:33.937116 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc038e9-5c29-4cb1-829f-e16e023e17a9 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:33.996264 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:34.042225 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc038e9-5c29-4cb1-829f-e16e023e17a9 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:34.077009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47409f69-792a-4b89-93f5-a5dc5a5a9b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:34.077691 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47409f69-792a-4b89-93f5-a5dc5a5a9b04 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:34.078009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47409f69-792a-4b89-93f5-a5dc5a5a9b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:34.078338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47409f69-792a-4b89-93f5-a5dc5a5a9b04 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:34.079098 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47409f69-792a-4b89-93f5-a5dc5a5a9b04 None admin] RBAC: Adding query filter params (name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:34.079402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47409f69-792a-4b89-93f5-a5dc5a5a9b04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:34.081564 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47409f69-792a-4b89-93f5-a5dc5a5a9b04 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:34.090125 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 460/927] 10.222.0.13 () {56 vars in 1103 bytes} [Sat Oct 22 23:58:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:58:34.097266 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79653f5b-7aa9-44f9-84b5-891913cd0bb7 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:34.206007 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79653f5b-7aa9-44f9-84b5-891913cd0bb7 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:34.241963 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceae10e-36e0-46ee-96e8-03bf46aa99a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:34.242582 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceae10e-36e0-46ee-96e8-03bf46aa99a1 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:34.242860 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceae10e-36e0-46ee-96e8-03bf46aa99a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:34.243133 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceae10e-36e0-46ee-96e8-03bf46aa99a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:34.254412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceae10e-36e0-46ee-96e8-03bf46aa99a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:34.255943 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceae10e-36e0-46ee-96e8-03bf46aa99a1 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:34.271244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-fceae10e-36e0-46ee-96e8-03bf46aa99a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:58:34.271244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:34.271244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:58:34.271244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:58:34.271244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:58:34.271244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:58:34.271244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:58:34.271244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:58:34.271244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:58:34.271244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:58:34.271244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:58:34.271244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:58:34.271244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:58:34.271244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:58:34.271244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:58:34.271244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:58:34.272278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:58:34.272278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:58:34.272278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:58:34.272278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:58:34.272278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:58:34.272278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:58:34.272278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:58:34.272278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:58:34.272278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:58:34.272278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:58:34.272278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:58:34.272278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:58:34.272278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:58:34.272278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:58:34.272278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:58:34.272278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:58:34.272278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:58:34.273211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:58:34.273211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:58:34.273211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:58:34.273211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 468/928] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:58:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:58:34.278331 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47409f69-792a-4b89-93f5-a5dc5a5a9b04 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:34.391643 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47409f69-792a-4b89-93f5-a5dc5a5a9b04 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:34.410251 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:34.428344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968ac61-6bb7-4364-8dbf-8e1c215bb6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:34.428981 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968ac61-6bb7-4364-8dbf-8e1c215bb6b1 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:34.429258 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968ac61-6bb7-4364-8dbf-8e1c215bb6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:34.429530 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968ac61-6bb7-4364-8dbf-8e1c215bb6b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:34.430167 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968ac61-6bb7-4364-8dbf-8e1c215bb6b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:34.430467 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968ac61-6bb7-4364-8dbf-8e1c215bb6b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:34.431842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968ac61-6bb7-4364-8dbf-8e1c215bb6b1 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:34.443874 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 461/929] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:58:34.450143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceae10e-36e0-46ee-96e8-03bf46aa99a1 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:34.509589 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:34.561545 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceae10e-36e0-46ee-96e8-03bf46aa99a1 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:34.598526 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95797270-5123-47cf-94b8-153e868a213c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:34.599253 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95797270-5123-47cf-94b8-153e868a213c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:34.599538 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95797270-5123-47cf-94b8-153e868a213c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:34.599813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95797270-5123-47cf-94b8-153e868a213c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:34.607562 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95797270-5123-47cf-94b8-153e868a213c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:34.609054 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95797270-5123-47cf-94b8-153e868a213c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:34.621744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-95797270-5123-47cf-94b8-153e868a213c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 22 23:58:34.621744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:34.621744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:58:34.621744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:58:34.621744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:58:34.621744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:58:34.621744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:58:34.621744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:58:34.621744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:58:34.621744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:58:34.621744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:58:34.621744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:58:34.621744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:58:34.621744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:58:34.621744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:58:34.621744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:58:34.623519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:58:34.623519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:58:34.623519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:58:34.623519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:58:34.623519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:58:34.623519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:58:34.623519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:58:34.623519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:58:34.623519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:58:34.623519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:58:34.623519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:58:34.623519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:58:34.623519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:58:34.623519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:58:34.623519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:58:34.623519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:58:34.623519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:58:34.623519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:58:34.625648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:58:34.625648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:58:34.625648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:58:34.625648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:58:34.625648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:58:34.625648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:58:34.625648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:58:34.625648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:58:34.625648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 22 23:58:34.625648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:58:34.625648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 469/930] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:58:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:58:34.631798 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968ac61-6bb7-4364-8dbf-8e1c215bb6b1 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:34.739163 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968ac61-6bb7-4364-8dbf-8e1c215bb6b1 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:34.776394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfef29-dbfd-45b7-be48-6a03a6d28e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:34.777017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfef29-dbfd-45b7-be48-6a03a6d28e13 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:34.777323 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfef29-dbfd-45b7-be48-6a03a6d28e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:34.777573 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfef29-dbfd-45b7-be48-6a03a6d28e13 None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:34.778336 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbfef29-dbfd-45b7-be48-6a03a6d28e13 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:34.778636 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbfef29-dbfd-45b7-be48-6a03a6d28e13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:34.780062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbfef29-dbfd-45b7-be48-6a03a6d28e13 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:34.797983 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 462/931] 10.222.0.13 () {56 vars in 1143 bytes} [Sat Oct 22 23:58:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:58:34.804548 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95797270-5123-47cf-94b8-153e868a213c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:34.912048 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95797270-5123-47cf-94b8-153e868a213c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:34.925745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:34.953864 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90820844-89e0-4be5-8bb4-d471e8a5beb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:34.954501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90820844-89e0-4be5-8bb4-d471e8a5beb6 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:34.954790 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90820844-89e0-4be5-8bb4-d471e8a5beb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:34.955046 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90820844-89e0-4be5-8bb4-d471e8a5beb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:35.026487 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90820844-89e0-4be5-8bb4-d471e8a5beb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:35.027916 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90820844-89e0-4be5-8bb4-d471e8a5beb6 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:35.138760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 470/932] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:58:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 22 23:58:35.144874 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfef29-dbfd-45b7-be48-6a03a6d28e13 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:35.180864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:35.250675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbfef29-dbfd-45b7-be48-6a03a6d28e13 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:35.289818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86eed8-7359-4969-a24a-1d4348a745d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:35.290379 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86eed8-7359-4969-a24a-1d4348a745d6 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:35.290662 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86eed8-7359-4969-a24a-1d4348a745d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:35.290934 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86eed8-7359-4969-a24a-1d4348a745d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:35.301509 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b86eed8-7359-4969-a24a-1d4348a745d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:35.302964 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b86eed8-7359-4969-a24a-1d4348a745d6 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:35.318276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-2b86eed8-7359-4969-a24a-1d4348a745d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:58:35.318276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:35.318276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:58:35.318276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:58:35.318276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:58:35.318276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:58:35.318276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:58:35.318276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:58:35.318276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:58:35.318276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:58:35.318276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:58:35.318276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:58:35.318276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:58:35.318276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:58:35.318276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:58:35.318276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:58:35.319680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:58:35.319680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:58:35.319680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:58:35.319680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:58:35.319680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:58:35.319680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:58:35.319680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:58:35.319680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:58:35.319680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:58:35.319680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:58:35.319680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:58:35.319680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:58:35.319680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:58:35.319680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:58:35.319680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:58:35.319680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:58:35.319680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:58:35.321113 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:58:35.321113 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:58:35.321113 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:58:35.321113 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 463/933] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:58:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:58:35.325800 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90820844-89e0-4be5-8bb4-d471e8a5beb6 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:35.339406 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:35.440740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90820844-89e0-4be5-8bb4-d471e8a5beb6 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:35.482659 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d5f8d-1277-4acb-b42c-ca7fd91263b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:35.483291 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d5f8d-1277-4acb-b42c-ca7fd91263b1 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:35.483637 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d5f8d-1277-4acb-b42c-ca7fd91263b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:35.483854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d5f8d-1277-4acb-b42c-ca7fd91263b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:35.484501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d5f8d-1277-4acb-b42c-ca7fd91263b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:35.484817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d5f8d-1277-4acb-b42c-ca7fd91263b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:35.486164 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d5f8d-1277-4acb-b42c-ca7fd91263b1 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:35.498730 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 471/934] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:58:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:58:35.505765 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86eed8-7359-4969-a24a-1d4348a745d6 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:35.603579 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:35.618908 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86eed8-7359-4969-a24a-1d4348a745d6 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:35.655641 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7064379-6170-4ee6-96bc-4401ee39cfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:35.656399 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7064379-6170-4ee6-96bc-4401ee39cfee None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:35.656710 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7064379-6170-4ee6-96bc-4401ee39cfee None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:35.657019 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7064379-6170-4ee6-96bc-4401ee39cfee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:35.665336 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7064379-6170-4ee6-96bc-4401ee39cfee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:35.667216 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7064379-6170-4ee6-96bc-4401ee39cfee None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:35.681164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-c7064379-6170-4ee6-96bc-4401ee39cfee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:58:35.681164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:35.681164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:58:35.681164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:58:35.681164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:58:35.681164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:58:35.681164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:58:35.681164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:58:35.681164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:58:35.681164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:58:35.681164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:58:35.681164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:58:35.681164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:58:35.681164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:58:35.681164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:58:35.682344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:58:35.682344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:58:35.682344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:58:35.682344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:58:35.682344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:58:35.682344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:58:35.682344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:58:35.682344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:58:35.682344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:58:35.682344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:58:35.682344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:58:35.682344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:58:35.682344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:58:35.682344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:58:35.682344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:58:35.682344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:58:35.682344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:58:35.683533 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:58:35.683533 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:58:35.683533 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:58:35.683533 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:58:35.683533 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:58:35.683533 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:58:35.683533 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:58:35.683533 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:58:35.688086 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d5f8d-1277-4acb-b42c-ca7fd91263b1 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:35.688550 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 464/935] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:58:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:58:35.766984 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:35.800293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d5f8d-1277-4acb-b42c-ca7fd91263b1 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:35.840030 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb7914-3b7c-49e6-bea8-d971be8d00ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:35.840990 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb7914-3b7c-49e6-bea8-d971be8d00ab None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:35.841408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb7914-3b7c-49e6-bea8-d971be8d00ab None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:35.841696 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb7914-3b7c-49e6-bea8-d971be8d00ab None admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:35.842590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb7914-3b7c-49e6-bea8-d971be8d00ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:35.843032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb7914-3b7c-49e6-bea8-d971be8d00ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:35.844783 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb7914-3b7c-49e6-bea8-d971be8d00ab None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:35.866032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 472/936] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Oct 22 23:58:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:58:35.867259 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7064379-6170-4ee6-96bc-4401ee39cfee None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:35.978173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7064379-6170-4ee6-96bc-4401ee39cfee None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:36.016583 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5173f-6505-42d5-93bd-05541086fb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:36.017188 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5173f-6505-42d5-93bd-05541086fb46 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:36.017507 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5173f-6505-42d5-93bd-05541086fb46 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:36.017787 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5173f-6505-42d5-93bd-05541086fb46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:36.019133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5173f-6505-42d5-93bd-05541086fb46 None admin] RBAC: Adding query filter params (role.id=73721a054c3842bba868ab52c1a3f8e0, scope.project.id=5c9246c059754504afee4711d113a3fe, user.id=24108b6355414f64ba5e7bf84011eee6) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:36.019440 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5173f-6505-42d5-93bd-05541086fb46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:36.020940 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5173f-6505-42d5-93bd-05541086fb46 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:36.026762 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 465/937] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:58:35 2022] GET /identity/v3/role_assignments?user.id=24108b6355414f64ba5e7bf84011eee6&scope.project.id=5c9246c059754504afee4711d113a3fe&role.id=73721a054c3842bba868ab52c1a3f8e0 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:58:38.281259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd46298-d07a-4fd9-a62d-94771acb0052 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:38.281797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd46298-d07a-4fd9-a62d-94771acb0052 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:38.281875 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd46298-d07a-4fd9-a62d-94771acb0052 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:38.283224 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 473/938] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:58:38.289996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8fbc5-dd92-49e6-8b4b-c8e2ddfe5fb9 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:38.290589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8fbc5-dd92-49e6-8b4b-c8e2ddfe5fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:38.290589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8fbc5-dd92-49e6-8b4b-c8e2ddfe5fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:38.310013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-3fa8fbc5-dd92-49e6-8b4b-c8e2ddfe5fb9 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:58:38.311261 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-3fa8fbc5-dd92-49e6-8b4b-c8e2ddfe5fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:58:38.314096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:38.388542 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 466/939] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:58:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:58:38.675224 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd46298-d07a-4fd9-a62d-94771acb0052 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:38.786125 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd46298-d07a-4fd9-a62d-94771acb0052 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:38.817859 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:38.824805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a2941-ca63-42fd-ad65-700267e2779d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:38.825567 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a2941-ca63-42fd-ad65-700267e2779d None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:38.825983 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a2941-ca63-42fd-ad65-700267e2779d None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:38.826319 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a2941-ca63-42fd-ad65-700267e2779d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:38.837674 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6a2941-ca63-42fd-ad65-700267e2779d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:58:38.842671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-cb6a2941-ca63-42fd-ad65-700267e2779d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:58:38.842671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:38.842671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:58:38.842671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:58:38.842671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:58:38.842671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:58:38.842671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:58:38.842671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:58:38.842671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:58:38.842671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:58:38.842671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:58:38.842671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:58:38.842671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:58:38.842671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:58:38.842671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:58:38.842671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:58:38.844516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:58:38.844516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:58:38.844516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:58:38.844516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:58:38.844516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:58:38.844516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:58:38.844516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:58:38.844516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:58:38.844516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:58:38.844516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:58:38.844516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:58:38.844516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:58:38.844516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:58:38.844516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:58:38.844516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:58:38.844516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:58:38.846822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:58:38.846822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:58:38.846822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:58:38.846822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:58:38.846822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:58:38.846822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:58:38.846822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:58:38.846822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:58:38.846822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:58:38.846822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:58:38.846822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:58:38.846822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:58:38.846822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 474/940] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Oct 22 23:58:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:58:38.852781 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8fbc5-dd92-49e6-8b4b-c8e2ddfe5fb9 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:38.969804 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8fbc5-dd92-49e6-8b4b-c8e2ddfe5fb9 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:39.006903 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700100c7-7883-48c5-be75-c0ba414a70f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:39.007540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700100c7-7883-48c5-be75-c0ba414a70f4 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:39.007823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700100c7-7883-48c5-be75-c0ba414a70f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:39.008122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700100c7-7883-48c5-be75-c0ba414a70f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:39.008911 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700100c7-7883-48c5-be75-c0ba414a70f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:39.009248 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700100c7-7883-48c5-be75-c0ba414a70f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:39.010798 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700100c7-7883-48c5-be75-c0ba414a70f4 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:39.019917 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 467/941] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:58:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:58:39.025681 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a2941-ca63-42fd-ad65-700267e2779d None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:39.143945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a2941-ca63-42fd-ad65-700267e2779d None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:39.182181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3f21b-2396-4aa6-8010-3e1460050a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:39.182938 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3f21b-2396-4aa6-8010-3e1460050a1e None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:39.183216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3f21b-2396-4aa6-8010-3e1460050a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:39.183488 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3f21b-2396-4aa6-8010-3e1460050a1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:39.194906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb3f21b-2396-4aa6-8010-3e1460050a1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:39.196525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb3f21b-2396-4aa6-8010-3e1460050a1e None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:39.214272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-3eb3f21b-2396-4aa6-8010-3e1460050a1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:58:39.214272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:39.214272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:58:39.214272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:58:39.214272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:58:39.214272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:58:39.214272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:58:39.214272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:58:39.214272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:58:39.214272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:58:39.214272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:58:39.214272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:58:39.214272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:58:39.214272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:58:39.214272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:58:39.214272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:58:39.215449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:58:39.215449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:58:39.215449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:58:39.215449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:58:39.215449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:58:39.215449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:58:39.215449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:58:39.215449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:58:39.215449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:58:39.215449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:58:39.215449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:58:39.215449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:58:39.215449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:58:39.215449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:58:39.215449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:58:39.215449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:58:39.215449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:58:39.216368 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:58:39.216368 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:58:39.216368 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:58:39.216368 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 475/942] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:58:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:58:39.221920 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700100c7-7883-48c5-be75-c0ba414a70f4 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:39.224759 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:39.333186 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700100c7-7883-48c5-be75-c0ba414a70f4 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:39.371104 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355a280-380b-4f47-98d2-e636bcdea857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:39.371706 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355a280-380b-4f47-98d2-e636bcdea857 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:39.372013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355a280-380b-4f47-98d2-e636bcdea857 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:39.372258 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355a280-380b-4f47-98d2-e636bcdea857 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:39.372928 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0355a280-380b-4f47-98d2-e636bcdea857 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:39.373222 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0355a280-380b-4f47-98d2-e636bcdea857 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:39.374578 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0355a280-380b-4f47-98d2-e636bcdea857 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:39.387183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 468/943] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:58:39.393655 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3f21b-2396-4aa6-8010-3e1460050a1e None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:39.441168 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:39.502606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3f21b-2396-4aa6-8010-3e1460050a1e None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:39.539797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f738fc-bf5a-4848-8f74-202f0e9a36a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:39.540665 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f738fc-bf5a-4848-8f74-202f0e9a36a2 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:39.541083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f738fc-bf5a-4848-8f74-202f0e9a36a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:39.541522 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f738fc-bf5a-4848-8f74-202f0e9a36a2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:39.550031 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f738fc-bf5a-4848-8f74-202f0e9a36a2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:39.551773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f738fc-bf5a-4848-8f74-202f0e9a36a2 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:39.563527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-b1f738fc-bf5a-4848-8f74-202f0e9a36a2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 22 23:58:39.563527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:39.563527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:58:39.563527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:58:39.563527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:58:39.563527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:58:39.563527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:58:39.563527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:58:39.563527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:58:39.563527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:58:39.563527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:58:39.563527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:58:39.563527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:58:39.563527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:58:39.563527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:58:39.563527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:58:39.564714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:58:39.564714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:58:39.564714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:58:39.564714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:58:39.564714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:58:39.564714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:58:39.564714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:58:39.564714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:58:39.564714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:58:39.564714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:58:39.564714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:58:39.564714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:58:39.564714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:58:39.564714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:58:39.564714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:58:39.564714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:58:39.564714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:58:39.564714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:58:39.566443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:58:39.566443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:58:39.566443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:58:39.566443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:58:39.566443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:58:39.566443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:58:39.566443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:58:39.566443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:58:39.566443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 22 23:58:39.566443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:58:39.566443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 476/944] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:58:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:58:39.571398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355a280-380b-4f47-98d2-e636bcdea857 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:39.637791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:39.682474 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355a280-380b-4f47-98d2-e636bcdea857 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:39.724318 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a9851-1d22-4f0a-aac1-d6b49682965b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:39.725317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a9851-1d22-4f0a-aac1-d6b49682965b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:39.725706 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a9851-1d22-4f0a-aac1-d6b49682965b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:39.726123 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a9851-1d22-4f0a-aac1-d6b49682965b None admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:39.727247 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a9851-1d22-4f0a-aac1-d6b49682965b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:39.727661 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a9851-1d22-4f0a-aac1-d6b49682965b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:39.729968 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a9851-1d22-4f0a-aac1-d6b49682965b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:39.755219 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 469/945] 10.222.0.13 () {56 vars in 1143 bytes} [Sat Oct 22 23:58:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:58:39.761561 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f738fc-bf5a-4848-8f74-202f0e9a36a2 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:39.875180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f738fc-bf5a-4848-8f74-202f0e9a36a2 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:39.876768 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:39.916895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750879d-1748-485e-94bd-3a2da7eaa11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:39.917616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750879d-1748-485e-94bd-3a2da7eaa11b None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:39.917937 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750879d-1748-485e-94bd-3a2da7eaa11b None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:39.918251 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750879d-1748-485e-94bd-3a2da7eaa11b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:39.931733 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750879d-1748-485e-94bd-3a2da7eaa11b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:39.933642 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750879d-1748-485e-94bd-3a2da7eaa11b None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:39.950274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-7750879d-1748-485e-94bd-3a2da7eaa11b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:58:39.950274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:39.950274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:58:39.950274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:58:39.950274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:58:39.950274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:58:39.950274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:58:39.950274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:58:39.950274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:58:39.950274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:58:39.950274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:58:39.950274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:58:39.950274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:58:39.950274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:58:39.950274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:58:39.950274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:58:39.951452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:58:39.951452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:58:39.951452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:58:39.951452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:58:39.951452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:58:39.951452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:58:39.951452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:58:39.951452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:58:39.951452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:58:39.951452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:58:39.951452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:58:39.951452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:58:39.951452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:58:39.951452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:58:39.951452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:58:39.951452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:58:39.951452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:58:39.952697 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:58:39.952697 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:58:39.952697 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:58:39.952697 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 477/946] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:58:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:58:39.957496 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a9851-1d22-4f0a-aac1-d6b49682965b None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:40.092053 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a9851-1d22-4f0a-aac1-d6b49682965b None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:40.118758 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:40.137116 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04363c2-927c-43bb-a465-4e59308d5a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:40.137735 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04363c2-927c-43bb-a465-4e59308d5a5e None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:40.138143 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04363c2-927c-43bb-a465-4e59308d5a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:40.138600 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04363c2-927c-43bb-a465-4e59308d5a5e None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:40.139360 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04363c2-927c-43bb-a465-4e59308d5a5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:40.139670 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04363c2-927c-43bb-a465-4e59308d5a5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:40.141115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04363c2-927c-43bb-a465-4e59308d5a5e None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:40.155758 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 470/947] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:58:40.162512 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750879d-1748-485e-94bd-3a2da7eaa11b None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:40.276570 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750879d-1748-485e-94bd-3a2da7eaa11b None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:40.311604 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8658e-1e1c-4a08-97a7-2be1cbc51fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:40.312229 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8658e-1e1c-4a08-97a7-2be1cbc51fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:40.312543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8658e-1e1c-4a08-97a7-2be1cbc51fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:40.312824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8658e-1e1c-4a08-97a7-2be1cbc51fa6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:40.321828 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8658e-1e1c-4a08-97a7-2be1cbc51fa6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:40.323505 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8658e-1e1c-4a08-97a7-2be1cbc51fa6 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:40.336881 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-5cd8658e-1e1c-4a08-97a7-2be1cbc51fa6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:58:40.336881 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:40.336881 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:58:40.336881 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:58:40.336881 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:58:40.336881 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:58:40.336881 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:58:40.336881 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:58:40.336881 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:58:40.336881 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:58:40.336881 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:58:40.336881 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:58:40.336881 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:58:40.336881 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:58:40.336881 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:58:40.337843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:58:40.337843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:58:40.337843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:58:40.337843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:58:40.337843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:58:40.337843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:58:40.337843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:58:40.337843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:58:40.337843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:58:40.337843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:58:40.337843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:58:40.337843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:58:40.337843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:58:40.337843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:58:40.337843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:58:40.337843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:58:40.337843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:58:40.338745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:58:40.338745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:58:40.338745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:58:40.338745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:58:40.338745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:58:40.338745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:58:40.338745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:58:40.338745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:58:40.338745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 478/948] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:58:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:58:40.344164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04363c2-927c-43bb-a465-4e59308d5a5e None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:40.459817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04363c2-927c-43bb-a465-4e59308d5a5e None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:40.496117 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2aed8-dc1b-4d9a-99e4-8c58eff0295b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:40.496752 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2aed8-dc1b-4d9a-99e4-8c58eff0295b None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:40.497033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2aed8-dc1b-4d9a-99e4-8c58eff0295b None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:40.497305 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2aed8-dc1b-4d9a-99e4-8c58eff0295b None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:40.498825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2aed8-dc1b-4d9a-99e4-8c58eff0295b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:40.499114 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2aed8-dc1b-4d9a-99e4-8c58eff0295b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:40.500546 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2aed8-dc1b-4d9a-99e4-8c58eff0295b None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:40.512582 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 471/949] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Oct 22 23:58:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:58:40.520132 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8658e-1e1c-4a08-97a7-2be1cbc51fa6 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:40.538541 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:40.638146 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8658e-1e1c-4a08-97a7-2be1cbc51fa6 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:40.679922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9ea3c-9b71-4a75-99f3-1d5bab3130a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:40.680624 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9ea3c-9b71-4a75-99f3-1d5bab3130a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:40.680972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9ea3c-9b71-4a75-99f3-1d5bab3130a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:40.681238 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9ea3c-9b71-4a75-99f3-1d5bab3130a9 None admin] PATH_INFO: `/v3/projects/5c9246c059754504afee4711d113a3fe/users/24108b6355414f64ba5e7bf84011eee6/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:40.683207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9ea3c-9b71-4a75-99f3-1d5bab3130a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c9246c059754504afee4711d113a3fe, user_id=24108b6355414f64ba5e7bf84011eee6, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:40.684651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9ea3c-9b71-4a75-99f3-1d5bab3130a9 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:40.700321 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 479/950] 10.222.0.13 () {58 vars in 1439 bytes} [Sat Oct 22 23:58:40 2022] PUT /identity/v3/projects/5c9246c059754504afee4711d113a3fe/users/24108b6355414f64ba5e7bf84011eee6/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 22 23:58:42.856909 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce74a3a-3248-4531-8db2-2a2b91712ca6 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:42.857441 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce74a3a-3248-4531-8db2-2a2b91712ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:42.857441 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce74a3a-3248-4531-8db2-2a2b91712ca6 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:42.858804 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 472/951] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:58:42.865359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d352b-5a2d-4afa-8b84-54118baedda1 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:42.865780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d352b-5a2d-4afa-8b84-54118baedda1 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:42.865901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d352b-5a2d-4afa-8b84-54118baedda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:42.885242 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-390d352b-5a2d-4afa-8b84-54118baedda1 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:58:42.886470 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-390d352b-5a2d-4afa-8b84-54118baedda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:58:42.956035 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 480/952] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:58:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:58:43.229980 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce74a3a-3248-4531-8db2-2a2b91712ca6 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:43.278614 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:43.339091 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce74a3a-3248-4531-8db2-2a2b91712ca6 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:43.378819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36635623-5fa3-451d-8333-4b7d858545c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:43.379385 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36635623-5fa3-451d-8333-4b7d858545c0 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:43.379693 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36635623-5fa3-451d-8333-4b7d858545c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:43.379967 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36635623-5fa3-451d-8333-4b7d858545c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:43.392093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36635623-5fa3-451d-8333-4b7d858545c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 22 23:58:43.397864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-36635623-5fa3-451d-8333-4b7d858545c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:58:43.397864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:43.397864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:58:43.397864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:58:43.397864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:58:43.397864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:58:43.397864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:58:43.397864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:58:43.397864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:58:43.397864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:58:43.397864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:58:43.397864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:58:43.397864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 22 23:58:43.397864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 22 23:58:43.397864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 22 23:58:43.397864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 22 23:58:43.398823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 22 23:58:43.398823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 22 23:58:43.398823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 22 23:58:43.398823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 22 23:58:43.398823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:58:43.398823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:58:43.398823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:58:43.398823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:58:43.398823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:58:43.398823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:58:43.398823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:58:43.398823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:58:43.398823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:58:43.398823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:58:43.398823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:58:43.398823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:58:43.399901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:58:43.399901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:58:43.399901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:58:43.399901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:58:43.399901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 22 23:58:43.399901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 22 23:58:43.399901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 22 23:58:43.399901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 22 23:58:43.399901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 22 23:58:43.399901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 22 23:58:43.399901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 22 23:58:43.399901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:58:43.399901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 473/953] 10.222.0.13 () {56 vars in 1096 bytes} [Sat Oct 22 23:58:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:58:43.405067 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d352b-5a2d-4afa-8b84-54118baedda1 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:43.408250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:43.515692 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d352b-5a2d-4afa-8b84-54118baedda1 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:43.551518 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cff3f-1b19-4071-85b8-2492f72677d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:43.552118 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cff3f-1b19-4071-85b8-2492f72677d9 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:43.552399 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cff3f-1b19-4071-85b8-2492f72677d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:43.552705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cff3f-1b19-4071-85b8-2492f72677d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:43.553360 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cff3f-1b19-4071-85b8-2492f72677d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:43.553683 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cff3f-1b19-4071-85b8-2492f72677d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:43.555020 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cff3f-1b19-4071-85b8-2492f72677d9 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:43.563249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 481/954] 10.222.0.13 () {56 vars in 1099 bytes} [Sat Oct 22 23:58:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:58:43.569716 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36635623-5fa3-451d-8333-4b7d858545c0 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:43.673532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:43.676746 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36635623-5fa3-451d-8333-4b7d858545c0 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:43.719911 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff8068-06a4-427f-a134-ae0a7a54d9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:43.720566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff8068-06a4-427f-a134-ae0a7a54d9e3 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:43.720855 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff8068-06a4-427f-a134-ae0a7a54d9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:43.721695 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff8068-06a4-427f-a134-ae0a7a54d9e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:43.732567 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff8068-06a4-427f-a134-ae0a7a54d9e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:43.735532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff8068-06a4-427f-a134-ae0a7a54d9e3 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:43.751581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-06ff8068-06a4-427f-a134-ae0a7a54d9e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:58:43.751581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:43.751581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:58:43.751581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:58:43.751581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:58:43.751581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:58:43.751581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:58:43.751581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:58:43.751581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:58:43.751581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:58:43.751581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:58:43.751581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:58:43.751581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:58:43.751581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:58:43.751581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:58:43.751581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:58:43.753502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:58:43.753502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:58:43.753502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:58:43.753502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:58:43.753502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:58:43.753502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:58:43.753502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:58:43.753502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:58:43.753502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:58:43.753502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:58:43.753502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:58:43.753502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:58:43.753502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:58:43.753502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:58:43.753502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:58:43.753502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:58:43.753502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:58:43.754592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:58:43.754592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:58:43.754592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:58:43.754592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 474/955] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:58:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 22 23:58:43.758810 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cff3f-1b19-4071-85b8-2492f72677d9 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:43.827205 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:43.865252 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cff3f-1b19-4071-85b8-2492f72677d9 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:43.901528 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb526829-20dd-4a16-8ddd-23f12a7ba9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:43.902251 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb526829-20dd-4a16-8ddd-23f12a7ba9fa None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:43.902416 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb526829-20dd-4a16-8ddd-23f12a7ba9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:43.902693 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb526829-20dd-4a16-8ddd-23f12a7ba9fa None admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:43.903350 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb526829-20dd-4a16-8ddd-23f12a7ba9fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:43.903660 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb526829-20dd-4a16-8ddd-23f12a7ba9fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:43.905087 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb526829-20dd-4a16-8ddd-23f12a7ba9fa None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:43.919051 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 482/956] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:58:43.925990 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff8068-06a4-427f-a134-ae0a7a54d9e3 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:44.032332 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff8068-06a4-427f-a134-ae0a7a54d9e3 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:44.068584 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629f63a-a99e-42d1-9c21-7ecc614bc0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:44.069228 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629f63a-a99e-42d1-9c21-7ecc614bc0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:44.069509 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629f63a-a99e-42d1-9c21-7ecc614bc0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:44.069782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629f63a-a99e-42d1-9c21-7ecc614bc0e2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:44.078350 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629f63a-a99e-42d1-9c21-7ecc614bc0e2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:44.079805 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629f63a-a99e-42d1-9c21-7ecc614bc0e2 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:44.092299 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-8629f63a-a99e-42d1-9c21-7ecc614bc0e2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 22 23:58:44.092299 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:44.092299 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:58:44.092299 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:58:44.092299 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:58:44.092299 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:58:44.092299 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:58:44.092299 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:58:44.092299 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:58:44.092299 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:58:44.092299 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:58:44.092299 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:58:44.092299 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 22 23:58:44.092299 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 22 23:58:44.092299 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 22 23:58:44.092299 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 22 23:58:44.093505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:58:44.093505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:58:44.093505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 22 23:58:44.093505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:58:44.093505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 22 23:58:44.093505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 22 23:58:44.093505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:58:44.093505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:58:44.093505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:58:44.093505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:58:44.093505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:58:44.093505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:58:44.093505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:58:44.093505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:58:44.093505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:58:44.093505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:58:44.093505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:58:44.093505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:58:44.094500 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:58:44.094500 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:58:44.094500 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 22 23:58:44.094500 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 22 23:58:44.094500 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 22 23:58:44.094500 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 22 23:58:44.094500 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 22 23:58:44.094500 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 22 23:58:44.094500 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 22 23:58:44.094500 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:58:44.094500 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 475/957] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:58:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:58:44.100561 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb526829-20dd-4a16-8ddd-23f12a7ba9fa None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:44.206043 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb526829-20dd-4a16-8ddd-23f12a7ba9fa None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:44.235286 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:44.245389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71795e8f-1181-4145-acf0-f9c8abea43eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:44.246010 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71795e8f-1181-4145-acf0-f9c8abea43eb None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:44.246328 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71795e8f-1181-4145-acf0-f9c8abea43eb None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:44.246606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71795e8f-1181-4145-acf0-f9c8abea43eb None admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:44.247438 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71795e8f-1181-4145-acf0-f9c8abea43eb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:44.247760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71795e8f-1181-4145-acf0-f9c8abea43eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:44.249337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71795e8f-1181-4145-acf0-f9c8abea43eb None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:44.271290 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 483/958] 10.222.0.13 () {56 vars in 1143 bytes} [Sat Oct 22 23:58:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:58:44.275422 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629f63a-a99e-42d1-9c21-7ecc614bc0e2 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:44.292520 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:44.383596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629f63a-a99e-42d1-9c21-7ecc614bc0e2 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:44.417762 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225987e7-0156-4134-b0b1-36a1d77962d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:44.418374 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225987e7-0156-4134-b0b1-36a1d77962d8 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:44.418666 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225987e7-0156-4134-b0b1-36a1d77962d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:44.418936 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225987e7-0156-4134-b0b1-36a1d77962d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:44.487877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225987e7-0156-4134-b0b1-36a1d77962d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:44.489278 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225987e7-0156-4134-b0b1-36a1d77962d8 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:44.503532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:44.588047 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 476/959] 10.222.0.13 () {58 vars in 1282 bytes} [Sat Oct 22 23:58:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 22 23:58:44.593884 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71795e8f-1181-4145-acf0-f9c8abea43eb None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:44.690156 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71795e8f-1181-4145-acf0-f9c8abea43eb None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:44.732020 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8ecee-b08b-42e4-ac19-712e25ca528c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:44.732811 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8ecee-b08b-42e4-ac19-712e25ca528c None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:44.733171 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8ecee-b08b-42e4-ac19-712e25ca528c None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:44.733475 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8ecee-b08b-42e4-ac19-712e25ca528c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:44.744537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb8ecee-b08b-42e4-ac19-712e25ca528c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:44.745910 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb8ecee-b08b-42e4-ac19-712e25ca528c None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:44.760066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-efb8ecee-b08b-42e4-ac19-712e25ca528c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:58:44.760066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:44.760066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:58:44.760066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:58:44.760066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:58:44.760066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:58:44.760066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:58:44.760066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:58:44.760066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:58:44.760066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:58:44.760066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:58:44.760066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:58:44.760066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 22 23:58:44.760066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 22 23:58:44.760066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 22 23:58:44.760066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 22 23:58:44.761218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:58:44.761218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:58:44.761218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:58:44.761218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:58:44.761218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:58:44.761218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:58:44.761218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:58:44.761218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:58:44.761218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:58:44.761218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:58:44.761218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:58:44.761218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:58:44.761218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:58:44.761218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:58:44.761218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:58:44.761218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:58:44.761218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 22 23:58:44.762336 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 22 23:58:44.762336 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 22 23:58:44.762336 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:58:44.762336 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 484/960] 10.222.0.13 () {56 vars in 1108 bytes} [Sat Oct 22 23:58:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:58:44.766889 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225987e7-0156-4134-b0b1-36a1d77962d8 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:44.863352 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225987e7-0156-4134-b0b1-36a1d77962d8 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:44.875829 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:44.897198 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377160f-0a8c-4f67-a7ec-73de6dda69e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:44.897793 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377160f-0a8c-4f67-a7ec-73de6dda69e3 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:44.898073 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377160f-0a8c-4f67-a7ec-73de6dda69e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:44.898345 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377160f-0a8c-4f67-a7ec-73de6dda69e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:44.898970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377160f-0a8c-4f67-a7ec-73de6dda69e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:44.899271 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377160f-0a8c-4f67-a7ec-73de6dda69e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:44.900643 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377160f-0a8c-4f67-a7ec-73de6dda69e3 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:44.912072 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 477/961] 10.222.0.13 () {56 vars in 1109 bytes} [Sat Oct 22 23:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:58:44.917755 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8ecee-b08b-42e4-ac19-712e25ca528c None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:44.951934 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:45.014854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8ecee-b08b-42e4-ac19-712e25ca528c None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:45.048065 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a2362-ccce-48f6-be01-a50471048f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:45.048717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a2362-ccce-48f6-be01-a50471048f2f None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:45.049004 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a2362-ccce-48f6-be01-a50471048f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:45.049323 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251a2362-ccce-48f6-be01-a50471048f2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:45.055740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a2362-ccce-48f6-be01-a50471048f2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:45.057196 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251a2362-ccce-48f6-be01-a50471048f2f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:45.068702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application [None req-251a2362-ccce-48f6-be01-a50471048f2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:58:45.068702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:45.068702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:58:45.068702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:58:45.068702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:58:45.068702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:58:45.068702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:58:45.068702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:58:45.068702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:58:45.068702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:58:45.068702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:58:45.068702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:58:45.068702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 22 23:58:45.068702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 22 23:58:45.068702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 22 23:58:45.069750 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 22 23:58:45.069750 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:58:45.069750 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:58:45.069750 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:58:45.069750 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:58:45.069750 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:58:45.069750 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:58:45.069750 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:58:45.069750 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application with Lock( Oct 22 23:58:45.069750 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:58:45.069750 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:58:45.069750 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:58:45.069750 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:58:45.069750 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:58:45.069750 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:58:45.069750 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:58:45.069750 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:58:45.070835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 22 23:58:45.070835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 22 23:58:45.070835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 22 23:58:45.070835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 22 23:58:45.070835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 22 23:58:45.070835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 22 23:58:45.070835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 22 23:58:45.070835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: ERROR keystone.server.flask.application Oct 22 23:58:45.070835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 485/962] 10.222.0.13 () {56 vars in 1111 bytes} [Sat Oct 22 23:58:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:58:45.075838 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377160f-0a8c-4f67-a7ec-73de6dda69e3 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:45.174235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377160f-0a8c-4f67-a7ec-73de6dda69e3 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:45.208696 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bd7324-5ac7-4fd4-9cdb-0b55404ca57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:45.209283 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd7324-5ac7-4fd4-9cdb-0b55404ca57a None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:45.209561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd7324-5ac7-4fd4-9cdb-0b55404ca57a None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:45.209834 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd7324-5ac7-4fd4-9cdb-0b55404ca57a None admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:45.210508 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd7324-5ac7-4fd4-9cdb-0b55404ca57a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:45.210810 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd7324-5ac7-4fd4-9cdb-0b55404ca57a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:45.212212 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bd7324-5ac7-4fd4-9cdb-0b55404ca57a None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:45.224008 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 478/963] 10.222.0.13 () {56 vars in 1148 bytes} [Sat Oct 22 23:58:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:58:45.230125 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a2362-ccce-48f6-be01-a50471048f2f None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:45.320523 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:45.326535 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251a2362-ccce-48f6-be01-a50471048f2f None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:45.359217 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ee7fd-471a-40a3-9f24-688074e13c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:45.359800 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ee7fd-471a-40a3-9f24-688074e13c54 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:45.360077 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ee7fd-471a-40a3-9f24-688074e13c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:45.360345 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ee7fd-471a-40a3-9f24-688074e13c54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:45.361078 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ee7fd-471a-40a3-9f24-688074e13c54 None admin] RBAC: Adding query filter params (role.id=839a55263e2c4cbaa2f945b75cf298d6, scope.project.id=5c9246c059754504afee4711d113a3fe, user.id=24108b6355414f64ba5e7bf84011eee6) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:45.361385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ee7fd-471a-40a3-9f24-688074e13c54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:45.362818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ee7fd-471a-40a3-9f24-688074e13c54 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:45.368367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 486/964] 10.222.0.13 () {56 vars in 1374 bytes} [Sat Oct 22 23:58:45 2022] GET /identity/v3/role_assignments?user.id=24108b6355414f64ba5e7bf84011eee6&scope.project.id=5c9246c059754504afee4711d113a3fe&role.id=839a55263e2c4cbaa2f945b75cf298d6 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:58:47.501880 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa265f-2930-4cc3-92f0-ae5401013337 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:47.502914 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa265f-2930-4cc3-92f0-ae5401013337 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:47.502914 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa265f-2930-4cc3-92f0-ae5401013337 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:47.504212 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 479/965] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:58:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:58:47.509457 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9782abe-fbd1-41b5-b2a1-6015409da93f None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:47.509874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9782abe-fbd1-41b5-b2a1-6015409da93f None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:47.509993 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9782abe-fbd1-41b5-b2a1-6015409da93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:47.528731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-b9782abe-fbd1-41b5-b2a1-6015409da93f None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:58:47.529903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b9782abe-fbd1-41b5-b2a1-6015409da93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:58:47.594561 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 487/966] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:58:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:58:47.880930 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa265f-2930-4cc3-92f0-ae5401013337 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:47.911992 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:47.979186 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa265f-2930-4cc3-92f0-ae5401013337 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:48.012174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee87741-5f6d-48a6-90f7-34ccec27ab57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:48.012818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee87741-5f6d-48a6-90f7-34ccec27ab57 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:48.013105 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee87741-5f6d-48a6-90f7-34ccec27ab57 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:48.013383 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee87741-5f6d-48a6-90f7-34ccec27ab57 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:48.013960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee87741-5f6d-48a6-90f7-34ccec27ab57 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:48.015314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee87741-5f6d-48a6-90f7-34ccec27ab57 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:48.024038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-dee87741-5f6d-48a6-90f7-34ccec27ab57 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 22 23:58:48.024038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:48.024038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:58:48.024038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:58:48.024038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:58:48.024038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:58:48.024038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:58:48.024038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:58:48.024038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 22 23:58:48.024038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:58:48.024038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:58:48.024038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:58:48.024038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:48.024038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:58:48.024038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:58:48.024038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:58:48.024038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:58:48.028754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:58:48.028754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:58:48.028754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:58:48.028754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:58:48.028754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:58:48.028754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:58:48.028754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:58:48.028754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:58:48.028754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:58:48.028754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:58:48.028754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:58:48.028754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:58:48.028754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:58:48.028754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:58:48.028754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:58:48.028754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:58:48.029921 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:58:48.029921 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:58:48.029921 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:58:48.029921 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:58:48.029921 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:58:48.029921 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:58:48.029921 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:58:48.029921 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:58:48.029921 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:58:48.029921 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:58:48.029921 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:58:48.029921 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:58:48.029921 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 22 23:58:48.029921 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:58:48.029921 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 480/967] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Oct 22 23:58:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:58:48.031073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9782abe-fbd1-41b5-b2a1-6015409da93f None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:48.121422 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:48.131754 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9782abe-fbd1-41b5-b2a1-6015409da93f None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:48.164835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0716a-f5e8-4e6c-825d-4cf4d972b9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:48.165429 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0716a-f5e8-4e6c-825d-4cf4d972b9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:48.165719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0716a-f5e8-4e6c-825d-4cf4d972b9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:48.165998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0716a-f5e8-4e6c-825d-4cf4d972b9d3 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:48.166616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c0716a-f5e8-4e6c-825d-4cf4d972b9d3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:48.166919 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c0716a-f5e8-4e6c-825d-4cf4d972b9d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:48.168266 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c0716a-f5e8-4e6c-825d-4cf4d972b9d3 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:48.172916 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 488/968] 10.222.0.13 () {56 vars in 1116 bytes} [Sat Oct 22 23:58:48 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 22 23:58:48.179695 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee87741-5f6d-48a6-90f7-34ccec27ab57 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:48.272914 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:48.281115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee87741-5f6d-48a6-90f7-34ccec27ab57 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:48.315535 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64234f-52da-46c0-a366-8ef96aaa2bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:48.316131 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64234f-52da-46c0-a366-8ef96aaa2bdb None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:48.316414 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64234f-52da-46c0-a366-8ef96aaa2bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:48.316720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64234f-52da-46c0-a366-8ef96aaa2bdb None admin] PATH_INFO: `/v3/services` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:48.317339 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce64234f-52da-46c0-a366-8ef96aaa2bdb None admin] RBAC: Adding query filter params (type=placement) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:48.317641 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce64234f-52da-46c0-a366-8ef96aaa2bdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:48.318997 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce64234f-52da-46c0-a366-8ef96aaa2bdb None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:48.323231 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 481/969] 10.222.0.13 () {56 vars in 1116 bytes} [Sat Oct 22 23:58:48 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:58:50.492247 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df5591-6486-4b81-b35d-0cc5f1f408b9 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:50.492815 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df5591-6486-4b81-b35d-0cc5f1f408b9 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:50.492815 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df5591-6486-4b81-b35d-0cc5f1f408b9 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:50.494123 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 489/970] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:58:50.499251 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048e991-a94a-40d0-aaff-f676d5ba4ac3 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:50.499647 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048e991-a94a-40d0-aaff-f676d5ba4ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:50.499811 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048e991-a94a-40d0-aaff-f676d5ba4ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:50.518707 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-2048e991-a94a-40d0-aaff-f676d5ba4ac3 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:58:50.519901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2048e991-a94a-40d0-aaff-f676d5ba4ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:58:50.585149 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 482/971] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:58:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:58:50.882854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df5591-6486-4b81-b35d-0cc5f1f408b9 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:50.980632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df5591-6486-4b81-b35d-0cc5f1f408b9 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:51.012373 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61153bb2-472f-4a93-a3d1-e09b5d5a162f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:51.012993 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61153bb2-472f-4a93-a3d1-e09b5d5a162f None admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:51.013274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61153bb2-472f-4a93-a3d1-e09b5d5a162f None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:51.013545 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61153bb2-472f-4a93-a3d1-e09b5d5a162f None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:51.014185 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61153bb2-472f-4a93-a3d1-e09b5d5a162f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:51.015555 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61153bb2-472f-4a93-a3d1-e09b5d5a162f None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:51.030016 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 490/972] 10.222.0.13 () {60 vars in 1141 bytes} [Sat Oct 22 23:58:50 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:58:53.174983 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da5a39-b97a-4cbe-a000-30c732dcd125 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:53.175524 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da5a39-b97a-4cbe-a000-30c732dcd125 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:53.175524 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da5a39-b97a-4cbe-a000-30c732dcd125 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:53.176894 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 483/973] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:58:53.181953 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd4e74-0e27-4348-994e-adf760e99449 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:53.182342 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd4e74-0e27-4348-994e-adf760e99449 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:53.182510 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd4e74-0e27-4348-994e-adf760e99449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:53.208007 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-d9bd4e74-0e27-4348-994e-adf760e99449 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:58:53.209229 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d9bd4e74-0e27-4348-994e-adf760e99449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:58:53.218384 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:53.280551 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 491/974] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:58:53 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:58:53.545639 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da5a39-b97a-4cbe-a000-30c732dcd125 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:53.632887 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:53.643135 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da5a39-b97a-4cbe-a000-30c732dcd125 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:53.677593 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34accdf7-e9bf-44e9-b373-621259002dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:53.678201 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34accdf7-e9bf-44e9-b373-621259002dca None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:53.678481 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34accdf7-e9bf-44e9-b373-621259002dca None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:53.678751 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34accdf7-e9bf-44e9-b373-621259002dca None admin] PATH_INFO: `/v3/services/placement` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:53.679321 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34accdf7-e9bf-44e9-b373-621259002dca None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:53.680698 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34accdf7-e9bf-44e9-b373-621259002dca None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:53.689589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-34accdf7-e9bf-44e9-b373-621259002dca None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 22 23:58:53.689589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:53.689589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:58:53.689589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:58:53.689589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:58:53.689589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:58:53.689589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:58:53.689589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:58:53.689589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 22 23:58:53.689589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:58:53.689589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:58:53.689589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:58:53.689589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:53.689589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:58:53.689589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:58:53.689589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:58:53.689589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:58:53.693127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:58:53.693127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:58:53.693127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:58:53.693127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:58:53.693127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:58:53.693127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:58:53.693127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:58:53.693127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:58:53.693127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:58:53.693127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:58:53.693127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:58:53.693127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:58:53.693127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:58:53.693127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:58:53.693127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:58:53.693127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:58:53.694183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:58:53.694183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:58:53.694183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:58:53.694183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:58:53.694183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:58:53.694183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:58:53.694183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:58:53.694183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:58:53.694183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:58:53.694183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:58:53.694183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:58:53.694183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:58:53.694183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 22 23:58:53.694183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:58:53.694183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 484/975] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Oct 22 23:58:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:58:53.697429 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd4e74-0e27-4348-994e-adf760e99449 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:53.801008 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd4e74-0e27-4348-994e-adf760e99449 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:53.835535 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68772e-f581-4a1c-b074-31e0b0581679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:53.836129 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68772e-f581-4a1c-b074-31e0b0581679 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:53.836408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68772e-f581-4a1c-b074-31e0b0581679 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:53.836709 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68772e-f581-4a1c-b074-31e0b0581679 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:53.837326 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68772e-f581-4a1c-b074-31e0b0581679 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:53.837627 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68772e-f581-4a1c-b074-31e0b0581679 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:53.838963 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68772e-f581-4a1c-b074-31e0b0581679 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:53.843819 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 492/976] 10.222.0.13 () {56 vars in 1116 bytes} [Sat Oct 22 23:58:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:58:53.850571 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34accdf7-e9bf-44e9-b373-621259002dca None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:53.945654 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34accdf7-e9bf-44e9-b373-621259002dca None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:53.978527 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ffae8c-cbc8-4dcc-9775-978d06ce78fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:53.979108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffae8c-cbc8-4dcc-9775-978d06ce78fa None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:53.979388 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffae8c-cbc8-4dcc-9775-978d06ce78fa None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:53.979659 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffae8c-cbc8-4dcc-9775-978d06ce78fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:53.980334 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ffae8c-cbc8-4dcc-9775-978d06ce78fa None admin] RBAC: Adding query filter params (interface=public, service_id=bff5f4fb955b4c7484c4f0f8451aaef9, region_id=RegionOne) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:53.980667 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ffae8c-cbc8-4dcc-9775-978d06ce78fa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:53.982005 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ffae8c-cbc8-4dcc-9775-978d06ce78fa None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:53.987008 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 485/977] 10.222.0.13 () {56 vars in 1251 bytes} [Sat Oct 22 23:58:53 2022] GET /identity/v3/endpoints?service_id=bff5f4fb955b4c7484c4f0f8451aaef9&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:58:53.992343 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68772e-f581-4a1c-b074-31e0b0581679 None admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:53.996062 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:54.091337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68772e-f581-4a1c-b074-31e0b0581679 None admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:54.126104 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6671ec8c-22d3-41a9-89b2-17d44762a347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:54.126701 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671ec8c-22d3-41a9-89b2-17d44762a347 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:54.126979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671ec8c-22d3-41a9-89b2-17d44762a347 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:54.127249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671ec8c-22d3-41a9-89b2-17d44762a347 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:54.128894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6671ec8c-22d3-41a9-89b2-17d44762a347 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:54.130233 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6671ec8c-22d3-41a9-89b2-17d44762a347 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:54.136857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 493/978] 10.222.0.13 () {56 vars in 1087 bytes} [Sat Oct 22 23:58:53 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 22 23:58:56.293702 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbde465-5a6c-42fc-a166-bacc89b11303 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:56.294216 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbde465-5a6c-42fc-a166-bacc89b11303 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:56.294322 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbde465-5a6c-42fc-a166-bacc89b11303 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:56.295608 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 486/979] 10.222.0.13 () {54 vars in 927 bytes} [Sat Oct 22 23:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 22 23:58:56.301240 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0e559-c708-4428-a70e-a096a6bd0030 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:56.301611 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0e559-c708-4428-a70e-a096a6bd0030 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:56.301809 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0e559-c708-4428-a70e-a096a6bd0030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:56.322480 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1da0e559-c708-4428-a70e-a096a6bd0030 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 22 23:58:56.323912 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1da0e559-c708-4428-a70e-a096a6bd0030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 22 23:58:56.383103 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:56.388781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 494/980] 10.222.0.13 () {58 vars in 1024 bytes} [Sat Oct 22 23:58:56 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 22 23:58:56.655737 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbde465-5a6c-42fc-a166-bacc89b11303 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:56.676136 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:56.767149 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbde465-5a6c-42fc-a166-bacc89b11303 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:56.801454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9e866-1bdb-4b94-ae72-fa2146d24fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:56.802069 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9e866-1bdb-4b94-ae72-fa2146d24fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:56.802347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9e866-1bdb-4b94-ae72-fa2146d24fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:56.802617 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9e866-1bdb-4b94-ae72-fa2146d24fe6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:56.803182 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9e866-1bdb-4b94-ae72-fa2146d24fe6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:56.804545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9e866-1bdb-4b94-ae72-fa2146d24fe6 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:56.812952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application [None req-58d9e866-1bdb-4b94-ae72-fa2146d24fe6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 22 23:58:56.812952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:56.812952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 22 23:58:56.812952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 22 23:58:56.812952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 22 23:58:56.812952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 22 23:58:56.812952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 22 23:58:56.812952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:58:56.812952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 22 23:58:56.812952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:58:56.812952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 22 23:58:56.812952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:58:56.812952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 22 23:58:56.812952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 22 23:58:56.812952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 22 23:58:56.812952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 22 23:58:56.812952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 22 23:58:56.814208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 22 23:58:56.814208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 22 23:58:56.814208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 22 23:58:56.814208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 22 23:58:56.814208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 22 23:58:56.814208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 22 23:58:56.814208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 22 23:58:56.814208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 22 23:58:56.814208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 22 23:58:56.814208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 22 23:58:56.814208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 22 23:58:56.814208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 22 23:58:56.814208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 22 23:58:56.814208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 22 23:58:56.814208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 22 23:58:56.814208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.get_or_create( Oct 22 23:58:56.815359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 22 23:58:56.815359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application with Lock( Oct 22 23:58:56.815359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 22 23:58:56.815359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self._enter() Oct 22 23:58:56.815359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 22 23:58:56.815359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 22 23:58:56.815359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 22 23:58:56.815359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application return self.creator() Oct 22 23:58:56.815359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 22 23:58:56.815359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application created_value = creator( Oct 22 23:58:56.815359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 22 23:58:56.815359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 22 23:58:56.815359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 22 23:58:56.815359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: ERROR keystone.server.flask.application Oct 22 23:58:56.815359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 487/981] 10.222.0.13 () {56 vars in 1117 bytes} [Sat Oct 22 23:58:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 22 23:58:56.820098 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0e559-c708-4428-a70e-a096a6bd0030 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:56.917135 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0e559-c708-4428-a70e-a096a6bd0030 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:56.949825 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c781a16-6399-4469-ad81-ffc20181fda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:56.950421 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c781a16-6399-4469-ad81-ffc20181fda3 None admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:56.950703 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c781a16-6399-4469-ad81-ffc20181fda3 None admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:56.950977 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c781a16-6399-4469-ad81-ffc20181fda3 None admin] PATH_INFO: `/v3/services` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:56.951592 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c781a16-6399-4469-ad81-ffc20181fda3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 22 23:58:56.951898 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c781a16-6399-4469-ad81-ffc20181fda3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:56.953269 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c781a16-6399-4469-ad81-ffc20181fda3 None admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:56.959022 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 495/982] 10.222.0.13 () {56 vars in 1116 bytes} [Sat Oct 22 23:58:56 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 22 23:58:56.964743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9e866-1bdb-4b94-ae72-fa2146d24fe6 None admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 22 23:58:57.044592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:58:57.062961 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9e866-1bdb-4b94-ae72-fa2146d24fe6 None admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 22 23:58:57.095620 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ea41a-2403-40ca-8630-45694d9a3608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 22 23:58:57.096765 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ea41a-2403-40ca-8630-45694d9a3608 None admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 22 23:58:57.097317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ea41a-2403-40ca-8630-45694d9a3608 None admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 22 23:58:57.097427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ea41a-2403-40ca-8630-45694d9a3608 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 22 23:58:57.098062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ea41a-2403-40ca-8630-45694d9a3608 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 22 23:58:57.099426 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ea41a-2403-40ca-8630-45694d9a3608 None admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 22 23:58:57.119801 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 488/983] 10.222.0.13 () {60 vars in 1144 bytes} [Sat Oct 22 23:58:56 2022] POST /identity/v3/endpoints => generated 331 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 22 23:59:06.955724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=69 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 22 23:59:07.115500 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:01.328158 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8a5ce-3b20-4cf1-835b-c3eed629fbc5 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:01.328750 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8a5ce-3b20-4cf1-835b-c3eed629fbc5 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:01.328750 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8a5ce-3b20-4cf1-835b-c3eed629fbc5 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:01.330253 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 496/984] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:00:01.336032 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a4909-a865-4ef7-884c-5b2e8da25359 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:01.336519 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a4909-a865-4ef7-884c-5b2e8da25359 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:01.336605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a4909-a865-4ef7-884c-5b2e8da25359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:01.442490 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-354a4909-a865-4ef7-884c-5b2e8da25359 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:00:01.445249 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-354a4909-a865-4ef7-884c-5b2e8da25359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:01.695199 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 489/985] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:00:02.000755 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458edfe8-a28a-4a80-a0a9-5a3fc5175c1e None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:02.001399 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458edfe8-a28a-4a80-a0a9-5a3fc5175c1e None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:02.001916 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458edfe8-a28a-4a80-a0a9-5a3fc5175c1e None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:02.003775 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 497/986] 10.222.0.13 () {54 vars in 964 bytes} [Sun Oct 23 00:00:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:00:02.010093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f177b3-42fb-4f16-912e-81ffb4df195e None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:02.010386 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f177b3-42fb-4f16-912e-81ffb4df195e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:02.010849 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f177b3-42fb-4f16-912e-81ffb4df195e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:02.093139 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-21f177b3-42fb-4f16-912e-81ffb4df195e None None] MFA Rules not processed for user `a09d45d080bc461c88f38ff281b75b06`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:00:02.094346 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-21f177b3-42fb-4f16-912e-81ffb4df195e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:02.116949 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:02.179381 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 490/987] 10.222.0.13 () {58 vars in 1061 bytes} [Sun Oct 23 00:00:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:00:02.188227 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458edfe8-a28a-4a80-a0a9-5a3fc5175c1e None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:00:02.190004 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-458edfe8-a28a-4a80-a0a9-5a3fc5175c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:02.341535 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458edfe8-a28a-4a80-a0a9-5a3fc5175c1e None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:00:02.439270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d8e24-c7d7-4a9c-aac4-6da6ed032a52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:00:02.439880 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d8e24-c7d7-4a9c-aac4-6da6ed032a52 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:02.440183 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d8e24-c7d7-4a9c-aac4-6da6ed032a52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:02.440995 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d8e24-c7d7-4a9c-aac4-6da6ed032a52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:02.442688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e88d8e24-c7d7-4a9c-aac4-6da6ed032a52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:02.504898 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:02.567131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d8e24-c7d7-4a9c-aac4-6da6ed032a52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:00:02.568464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d8e24-c7d7-4a9c-aac4-6da6ed032a52 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:00:02.589396 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 498/988] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:00:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:00:05.097075 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4760ba-5e9f-450a-8799-ef50590649e9 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:05.097652 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4760ba-5e9f-450a-8799-ef50590649e9 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:05.097652 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4760ba-5e9f-450a-8799-ef50590649e9 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:05.099018 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 491/989] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:00:05.104373 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e3eec-8155-41d1-939c-3ee64cd9c9f1 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:05.105166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e3eec-8155-41d1-939c-3ee64cd9c9f1 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:05.105636 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e3eec-8155-41d1-939c-3ee64cd9c9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:05.195852 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-9e1e3eec-8155-41d1-939c-3ee64cd9c9f1 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:00:05.197108 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9e1e3eec-8155-41d1-939c-3ee64cd9c9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:05.293279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 499/990] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:00:05.575372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4760ba-5e9f-450a-8799-ef50590649e9 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:00:05.577242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-1f4760ba-5e9f-450a-8799-ef50590649e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:05.684728 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4760ba-5e9f-450a-8799-ef50590649e9 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:00:05.756101 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:05.758551 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e06ffa-539b-407b-8a00-dbd715ce94a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:00:05.759359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e06ffa-539b-407b-8a00-dbd715ce94a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:05.759709 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e06ffa-539b-407b-8a00-dbd715ce94a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:05.760138 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e06ffa-539b-407b-8a00-dbd715ce94a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:05.761980 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-39e06ffa-539b-407b-8a00-dbd715ce94a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:05.870288 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e06ffa-539b-407b-8a00-dbd715ce94a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:00:05.871782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e06ffa-539b-407b-8a00-dbd715ce94a8 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:00:05.889724 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 492/991] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:00:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:00:08.360159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f063c75-f92d-4a48-9bc2-6ac250611026 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:08.360733 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f063c75-f92d-4a48-9bc2-6ac250611026 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:08.360733 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f063c75-f92d-4a48-9bc2-6ac250611026 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:08.362008 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 500/992] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:00:08.368350 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b9c67-3a35-4797-a1f9-064b4181673a None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:08.368801 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b9c67-3a35-4797-a1f9-064b4181673a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:08.368937 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b9c67-3a35-4797-a1f9-064b4181673a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:08.434380 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-0b5b9c67-3a35-4797-a1f9-064b4181673a None None] MFA Rules not processed for user `214a39c2cb9344d8bab53fdc6b5324b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:00:08.435577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-0b5b9c67-3a35-4797-a1f9-064b4181673a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:08.485811 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:08.529424 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 493/993] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:00:08 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:00:08.811457 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f063c75-f92d-4a48-9bc2-6ac250611026 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:00:08.813047 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9f063c75-f92d-4a48-9bc2-6ac250611026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:08.824324 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:08.915986 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f063c75-f92d-4a48-9bc2-6ac250611026 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:00:08.988997 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ead34-1624-44dc-8b5c-832eb4f8ae6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:00:08.989602 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ead34-1624-44dc-8b5c-832eb4f8ae6e service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:08.989888 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ead34-1624-44dc-8b5c-832eb4f8ae6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:08.990169 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ead34-1624-44dc-8b5c-832eb4f8ae6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:08.991751 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ce5ead34-1624-44dc-8b5c-832eb4f8ae6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:09.053296 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:09.100922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ead34-1624-44dc-8b5c-832eb4f8ae6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:00:09.102361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ead34-1624-44dc-8b5c-832eb4f8ae6e service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:00:09.123298 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 501/994] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:00:08 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:00:11.798495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b855d72-adeb-44bc-b72a-359788ad7d7f None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:11.799023 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b855d72-adeb-44bc-b72a-359788ad7d7f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:11.799023 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b855d72-adeb-44bc-b72a-359788ad7d7f None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:11.801002 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 494/995] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:00:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:00:11.806071 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300cff3-966f-46c2-91da-31f2e7526811 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:11.806492 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300cff3-966f-46c2-91da-31f2e7526811 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:11.806603 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300cff3-966f-46c2-91da-31f2e7526811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:11.871810 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-0300cff3-966f-46c2-91da-31f2e7526811 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:00:11.873050 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-0300cff3-966f-46c2-91da-31f2e7526811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:11.968413 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 502/996] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:00:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:00:12.253376 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b855d72-adeb-44bc-b72a-359788ad7d7f None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:00:12.254950 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-6b855d72-adeb-44bc-b72a-359788ad7d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:12.355333 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b855d72-adeb-44bc-b72a-359788ad7d7f None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:00:12.426656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0fb11d-07b7-42a1-9e35-709b6727c6bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:00:12.427184 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fb11d-07b7-42a1-9e35-709b6727c6bd service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:12.427464 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fb11d-07b7-42a1-9e35-709b6727c6bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:12.427736 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fb11d-07b7-42a1-9e35-709b6727c6bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:12.429344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ed0fb11d-07b7-42a1-9e35-709b6727c6bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:12.435966 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:12.537245 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0fb11d-07b7-42a1-9e35-709b6727c6bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:00:12.538787 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0fb11d-07b7-42a1-9e35-709b6727c6bd service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:00:12.558770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 495/997] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:00:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:00:15.340777 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463546e-ed55-4264-a7eb-26695007ff12 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:15.341302 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463546e-ed55-4264-a7eb-26695007ff12 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:15.341302 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463546e-ed55-4264-a7eb-26695007ff12 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:15.342669 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 503/998] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:00:15.347963 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a6b20-9093-4267-8267-4f91ed499e81 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:15.348398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a6b20-9093-4267-8267-4f91ed499e81 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:15.348548 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a6b20-9093-4267-8267-4f91ed499e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:15.418752 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-474a6b20-9093-4267-8267-4f91ed499e81 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:00:15.419901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-474a6b20-9093-4267-8267-4f91ed499e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:15.482993 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:15.526564 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 496/999] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:00:15.831687 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1910ccd-1542-45a6-ae36-362786db56a1 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:15.831889 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1910ccd-1542-45a6-ae36-362786db56a1 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:15.832155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1910ccd-1542-45a6-ae36-362786db56a1 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:15.833570 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 504/1000] 10.222.0.13 () {54 vars in 964 bytes} [Sun Oct 23 00:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:00:15.839486 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c4409-f19b-43e6-8450-0c00eea75e37 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:15.839841 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c4409-f19b-43e6-8450-0c00eea75e37 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:15.840185 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c4409-f19b-43e6-8450-0c00eea75e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:15.915936 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-8c5c4409-f19b-43e6-8450-0c00eea75e37 None None] MFA Rules not processed for user `a09d45d080bc461c88f38ff281b75b06`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:00:15.917174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8c5c4409-f19b-43e6-8450-0c00eea75e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:15.991534 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 497/1001] 10.222.0.13 () {58 vars in 1061 bytes} [Sun Oct 23 00:00:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:00:16.000080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1910ccd-1542-45a6-ae36-362786db56a1 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:00:16.001655 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d1910ccd-1542-45a6-ae36-362786db56a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:16.021501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:16.107858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1910ccd-1542-45a6-ae36-362786db56a1 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:00:16.183595 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9d4b1-59a5-449c-9a05-455235441b62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:00:16.184202 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9d4b1-59a5-449c-9a05-455235441b62 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:16.184502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9d4b1-59a5-449c-9a05-455235441b62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:16.184784 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9d4b1-59a5-449c-9a05-455235441b62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:16.186359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1dc9d4b1-59a5-449c-9a05-455235441b62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:16.257569 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:16.296374 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9d4b1-59a5-449c-9a05-455235441b62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:00:16.297829 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc9d4b1-59a5-449c-9a05-455235441b62 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:00:16.315890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 505/1002] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:00:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:00:18.778543 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b208ed7-a046-4532-95a6-02a4ca657c35 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:18.779058 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b208ed7-a046-4532-95a6-02a4ca657c35 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:18.779058 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b208ed7-a046-4532-95a6-02a4ca657c35 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:18.780339 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 498/1003] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:00:18.786249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0b797-7fe6-4d60-a56f-829276fc6f9e None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:18.786641 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0b797-7fe6-4d60-a56f-829276fc6f9e None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:18.786816 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0b797-7fe6-4d60-a56f-829276fc6f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:18.855345 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-8ec0b797-7fe6-4d60-a56f-829276fc6f9e None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:00:18.856570 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-8ec0b797-7fe6-4d60-a56f-829276fc6f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:18.953356 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 506/1004] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:00:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:00:19.241578 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b208ed7-a046-4532-95a6-02a4ca657c35 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:00:19.243201 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-5b208ed7-a046-4532-95a6-02a4ca657c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:19.305378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:19.350054 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b208ed7-a046-4532-95a6-02a4ca657c35 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:00:19.424009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da901daa-a0c6-4846-9648-ab0bb2f732dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:00:19.424580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da901daa-a0c6-4846-9648-ab0bb2f732dc service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:19.424935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da901daa-a0c6-4846-9648-ab0bb2f732dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:19.425206 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da901daa-a0c6-4846-9648-ab0bb2f732dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:19.426779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-da901daa-a0c6-4846-9648-ab0bb2f732dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:19.533809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:19.536419 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da901daa-a0c6-4846-9648-ab0bb2f732dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:00:19.538341 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da901daa-a0c6-4846-9648-ab0bb2f732dc service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:00:19.556733 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 499/1005] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:00:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:00:21.796700 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e63b9-5363-437f-99ba-8b32f6b326d6 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:21.797276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e63b9-5363-437f-99ba-8b32f6b326d6 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:21.797276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e63b9-5363-437f-99ba-8b32f6b326d6 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:21.798585 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 507/1006] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:00:21.805254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064738f-4172-49b2-9777-832d2c3d2d63 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:21.805678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064738f-4172-49b2-9777-832d2c3d2d63 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:21.805792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064738f-4172-49b2-9777-832d2c3d2d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:21.879093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-0064738f-4172-49b2-9777-832d2c3d2d63 None None] MFA Rules not processed for user `214a39c2cb9344d8bab53fdc6b5324b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:00:21.880668 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-0064738f-4172-49b2-9777-832d2c3d2d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:21.983000 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 500/1007] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:00:21 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:00:22.263511 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e63b9-5363-437f-99ba-8b32f6b326d6 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:00:22.265156 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-0f8e63b9-5363-437f-99ba-8b32f6b326d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:22.297052 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:22.371375 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e63b9-5363-437f-99ba-8b32f6b326d6 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:00:22.448240 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1ccdd-5ea8-4035-97de-07c1ebadfe5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:00:22.448891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1ccdd-5ea8-4035-97de-07c1ebadfe5a service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:22.449174 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1ccdd-5ea8-4035-97de-07c1ebadfe5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:22.449447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1ccdd-5ea8-4035-97de-07c1ebadfe5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:22.451062 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a2b1ccdd-5ea8-4035-97de-07c1ebadfe5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:22.531634 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:22.564791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1ccdd-5ea8-4035-97de-07c1ebadfe5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:00:22.566918 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1ccdd-5ea8-4035-97de-07c1ebadfe5a service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:00:22.585905 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 508/1008] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:00:22 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:00:24.949568 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9f3d8-7bcf-4b8c-9bbf-0dd4d49e5f1a None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:24.950131 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9f3d8-7bcf-4b8c-9bbf-0dd4d49e5f1a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:24.950131 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9f3d8-7bcf-4b8c-9bbf-0dd4d49e5f1a None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:24.951447 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 501/1009] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:00:24.958169 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3025d8-6e83-4f0c-a85e-b80a2f451704 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:24.958605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3025d8-6e83-4f0c-a85e-b80a2f451704 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:24.958705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3025d8-6e83-4f0c-a85e-b80a2f451704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:25.023701 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-3b3025d8-6e83-4f0c-a85e-b80a2f451704 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:00:25.024914 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-3b3025d8-6e83-4f0c-a85e-b80a2f451704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:25.118145 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 509/1010] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:00:25.401980 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9f3d8-7bcf-4b8c-9bbf-0dd4d49e5f1a None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:00:25.403529 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-53a9f3d8-7bcf-4b8c-9bbf-0dd4d49e5f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:25.473930 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:25.507865 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a9f3d8-7bcf-4b8c-9bbf-0dd4d49e5f1a None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:00:25.586565 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc9d52-89eb-4e41-833d-e9ad37af1978 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:00:25.587168 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc9d52-89eb-4e41-833d-e9ad37af1978 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:25.587447 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc9d52-89eb-4e41-833d-e9ad37af1978 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:25.587720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc9d52-89eb-4e41-833d-e9ad37af1978 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:25.589341 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-66dc9d52-89eb-4e41-833d-e9ad37af1978 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:25.702049 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dc9d52-89eb-4e41-833d-e9ad37af1978 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:00:25.703480 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dc9d52-89eb-4e41-833d-e9ad37af1978 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:00:25.714347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:25.723309 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 502/1011] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:00:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:00:27.967128 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad671e-00db-4191-936f-4ed5c20eee9f None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:27.967701 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad671e-00db-4191-936f-4ed5c20eee9f None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:27.967701 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad671e-00db-4191-936f-4ed5c20eee9f None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:27.969052 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 510/1012] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:00:27.975302 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d77bf6-5cda-40bf-bdee-95032b79c4c8 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:27.975715 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d77bf6-5cda-40bf-bdee-95032b79c4c8 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:27.975863 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d77bf6-5cda-40bf-bdee-95032b79c4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:28.040054 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-33d77bf6-5cda-40bf-bdee-95032b79c4c8 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:00:28.041285 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-33d77bf6-5cda-40bf-bdee-95032b79c4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:28.137451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 503/1013] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:00:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:00:28.424287 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad671e-00db-4191-936f-4ed5c20eee9f None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:00:28.426416 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-17ad671e-00db-4191-936f-4ed5c20eee9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:28.466133 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:28.530078 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad671e-00db-4191-936f-4ed5c20eee9f None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:00:28.611544 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770908f0-843e-4118-9910-3e9a7e43f13f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:00:28.612201 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770908f0-843e-4118-9910-3e9a7e43f13f service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:28.613052 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770908f0-843e-4118-9910-3e9a7e43f13f service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:28.613328 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770908f0-843e-4118-9910-3e9a7e43f13f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:28.614923 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-770908f0-843e-4118-9910-3e9a7e43f13f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:28.711233 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:28.739588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770908f0-843e-4118-9910-3e9a7e43f13f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:00:28.741920 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770908f0-843e-4118-9910-3e9a7e43f13f service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:00:28.763475 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 511/1014] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:00:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:00:31.269275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585fb9c-d599-4c5f-bdce-7d4111aac22c None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:31.269786 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585fb9c-d599-4c5f-bdce-7d4111aac22c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:31.269900 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585fb9c-d599-4c5f-bdce-7d4111aac22c None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:31.271168 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 504/1015] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:00:31.276351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ffb10-dddf-402f-9968-9089067306ef None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:31.276753 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ffb10-dddf-402f-9968-9089067306ef None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:31.276957 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ffb10-dddf-402f-9968-9089067306ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:31.343307 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-9d3ffb10-dddf-402f-9968-9089067306ef None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:00:31.344543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9d3ffb10-dddf-402f-9968-9089067306ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:31.441162 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 512/1016] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:00:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:00:31.726812 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585fb9c-d599-4c5f-bdce-7d4111aac22c None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:00:31.728387 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c585fb9c-d599-4c5f-bdce-7d4111aac22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:31.805520 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:31.833578 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585fb9c-d599-4c5f-bdce-7d4111aac22c None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:00:31.906485 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb569bf2-4ca1-443c-832d-82d9d667bfe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:00:31.907086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb569bf2-4ca1-443c-832d-82d9d667bfe5 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:31.907365 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb569bf2-4ca1-443c-832d-82d9d667bfe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:31.907634 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb569bf2-4ca1-443c-832d-82d9d667bfe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:31.910223 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-fb569bf2-4ca1-443c-832d-82d9d667bfe5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:32.015872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb569bf2-4ca1-443c-832d-82d9d667bfe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:00:32.017245 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb569bf2-4ca1-443c-832d-82d9d667bfe5 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:00:32.036323 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 505/1017] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:00:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:00:36.152289 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ec86e-5642-4d38-a1b9-ccd3da60e6a3 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:36.152826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ec86e-5642-4d38-a1b9-ccd3da60e6a3 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:36.152826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ec86e-5642-4d38-a1b9-ccd3da60e6a3 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:36.154181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 513/1018] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:00:36.160448 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be71d52-953f-44db-9001-3a77e5c42422 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:36.160897 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be71d52-953f-44db-9001-3a77e5c42422 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:36.160990 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be71d52-953f-44db-9001-3a77e5c42422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:36.166542 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:36.228889 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-1be71d52-953f-44db-9001-3a77e5c42422 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:00:36.230075 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-1be71d52-953f-44db-9001-3a77e5c42422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:36.331745 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 506/1019] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:00:36.611218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ec86e-5642-4d38-a1b9-ccd3da60e6a3 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:00:36.613301 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-130ec86e-5642-4d38-a1b9-ccd3da60e6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:36.662937 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:36.719251 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ec86e-5642-4d38-a1b9-ccd3da60e6a3 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:00:36.793087 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51579754-9445-4fbd-b0de-f6fd7b82157e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:00:36.793690 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51579754-9445-4fbd-b0de-f6fd7b82157e service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:36.793970 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51579754-9445-4fbd-b0de-f6fd7b82157e service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:36.794238 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51579754-9445-4fbd-b0de-f6fd7b82157e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:36.795809 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-51579754-9445-4fbd-b0de-f6fd7b82157e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:36.888579 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:36.904960 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51579754-9445-4fbd-b0de-f6fd7b82157e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:00:36.906389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51579754-9445-4fbd-b0de-f6fd7b82157e service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:00:36.925098 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 514/1020] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:00:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:00:39.736899 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc070add-967d-44bd-b812-c7b0cc9f8629 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:39.737902 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc070add-967d-44bd-b812-c7b0cc9f8629 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:39.737902 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc070add-967d-44bd-b812-c7b0cc9f8629 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:39.738959 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 507/1021] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:00:39.744170 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d892da-4b3f-4645-9274-34c9ae9925dd None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:39.744636 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d892da-4b3f-4645-9274-34c9ae9925dd None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:39.744894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d892da-4b3f-4645-9274-34c9ae9925dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:39.813214 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-64d892da-4b3f-4645-9274-34c9ae9925dd None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:00:39.814430 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-64d892da-4b3f-4645-9274-34c9ae9925dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:39.916548 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 515/1022] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:00:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:00:40.208946 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc070add-967d-44bd-b812-c7b0cc9f8629 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:00:40.210486 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-fc070add-967d-44bd-b812-c7b0cc9f8629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:40.304197 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:40.321710 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc070add-967d-44bd-b812-c7b0cc9f8629 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:00:40.396141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ea0f6-f8bb-4d15-a59d-8c4ebb15ace4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:00:40.396687 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ea0f6-f8bb-4d15-a59d-8c4ebb15ace4 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:40.396966 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ea0f6-f8bb-4d15-a59d-8c4ebb15ace4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:40.397235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ea0f6-f8bb-4d15-a59d-8c4ebb15ace4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:40.398814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-fa3ea0f6-f8bb-4d15-a59d-8c4ebb15ace4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:40.508676 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ea0f6-f8bb-4d15-a59d-8c4ebb15ace4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:00:40.510092 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3ea0f6-f8bb-4d15-a59d-8c4ebb15ace4 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:00:40.527633 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 508/1023] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:00:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:00:44.658527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725176a4-0599-4aa0-b186-0ba90b0c0506 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:44.659010 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725176a4-0599-4aa0-b186-0ba90b0c0506 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:44.659256 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725176a4-0599-4aa0-b186-0ba90b0c0506 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:44.660748 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 516/1024] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:00:44.665971 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7048f-8f5a-429e-b9e3-93f225ebf5dd None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:44.666374 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7048f-8f5a-429e-b9e3-93f225ebf5dd None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:44.666595 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7048f-8f5a-429e-b9e3-93f225ebf5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:44.685102 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:44.740753 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-94b7048f-8f5a-429e-b9e3-93f225ebf5dd None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:00:44.741975 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-94b7048f-8f5a-429e-b9e3-93f225ebf5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:44.840214 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 509/1025] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:00:45.123273 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725176a4-0599-4aa0-b186-0ba90b0c0506 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:00:45.124856 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-725176a4-0599-4aa0-b186-0ba90b0c0506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:45.182743 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:45.233460 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725176a4-0599-4aa0-b186-0ba90b0c0506 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:00:45.311381 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054fa77f-e2c5-427f-af24-a9b1ad9387a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:00:45.312303 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fa77f-e2c5-427f-af24-a9b1ad9387a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:45.312303 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fa77f-e2c5-427f-af24-a9b1ad9387a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:45.312542 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fa77f-e2c5-427f-af24-a9b1ad9387a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:45.314131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-054fa77f-e2c5-427f-af24-a9b1ad9387a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:45.426309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:45.432039 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054fa77f-e2c5-427f-af24-a9b1ad9387a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:00:45.433871 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054fa77f-e2c5-427f-af24-a9b1ad9387a8 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:00:45.453371 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 517/1026] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:00:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:00:50.379613 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568479f6-8767-4c9e-9550-c919b302496e None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:50.380048 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568479f6-8767-4c9e-9550-c919b302496e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:50.380152 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568479f6-8767-4c9e-9550-c919b302496e None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:50.385956 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 510/1027] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 00:00:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:00:50.392838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9b142-386c-4e67-a836-3bc0535abfaf None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:50.393636 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9b142-386c-4e67-a836-3bc0535abfaf None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:50.394082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9b142-386c-4e67-a836-3bc0535abfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:50.538379 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-59e9b142-386c-4e67-a836-3bc0535abfaf None None] MFA Rules not processed for user `24108b6355414f64ba5e7bf84011eee6`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:00:50.539941 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-59e9b142-386c-4e67-a836-3bc0535abfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:50.678753 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 518/1028] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 00:00:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:00:54.838630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=59 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:54.930530 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772b12d-7b25-4638-b57f-85ecddc87fec None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:54.930820 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772b12d-7b25-4638-b57f-85ecddc87fec None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:54.931095 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6772b12d-7b25-4638-b57f-85ecddc87fec None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:54.932774 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 511/1029] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 00:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:00:54.938551 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e15ac2-c529-4661-99f1-157ee0a325c8 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:54.939023 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e15ac2-c529-4661-99f1-157ee0a325c8 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:54.939159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e15ac2-c529-4661-99f1-157ee0a325c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:54.994903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:55.019522 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-a3e15ac2-c529-4661-99f1-157ee0a325c8 None None] MFA Rules not processed for user `24108b6355414f64ba5e7bf84011eee6`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:00:55.020760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a3e15ac2-c529-4661-99f1-157ee0a325c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:55.142664 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 519/1030] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 00:00:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:00:56.437256 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578006b-ae09-4449-97e4-0a16fee36015 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:56.437757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578006b-ae09-4449-97e4-0a16fee36015 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:56.437757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578006b-ae09-4449-97e4-0a16fee36015 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:56.439107 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 512/1031] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 00:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:00:56.444894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3b3d9-71bb-489d-8a8c-cada00016ab5 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:00:56.445349 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3b3d9-71bb-489d-8a8c-cada00016ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:00:56.445456 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3b3d9-71bb-489d-8a8c-cada00016ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:00:56.523247 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1be3b3d9-71bb-489d-8a8c-cada00016ab5 None None] MFA Rules not processed for user `24108b6355414f64ba5e7bf84011eee6`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:00:56.524464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1be3b3d9-71bb-489d-8a8c-cada00016ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:00:56.568030 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:00:56.635453 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 520/1032] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 00:00:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:01:00.151792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e3739-bee9-47c4-a2a6-18b49416cc19 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:00.152346 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e3739-bee9-47c4-a2a6-18b49416cc19 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:00.152346 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275e3739-bee9-47c4-a2a6-18b49416cc19 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:00.153872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 513/1033] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:01:00.159439 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48c75e-832e-44e4-9b48-8da811a45f86 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:00.159863 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48c75e-832e-44e4-9b48-8da811a45f86 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:00.160131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48c75e-832e-44e4-9b48-8da811a45f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:00.238664 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-ad48c75e-832e-44e4-9b48-8da811a45f86 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:00.240010 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ad48c75e-832e-44e4-9b48-8da811a45f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:00.347328 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 521/1034] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:01:00.649003 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d636d-1f82-4b61-be39-5dcfdb8cce93 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:00.649737 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d636d-1f82-4b61-be39-5dcfdb8cce93 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:00.650628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d636d-1f82-4b61-be39-5dcfdb8cce93 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:00.655498 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 514/1035] 10.222.0.13 () {54 vars in 961 bytes} [Sun Oct 23 00:01:00 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:01:00.674172 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da513b-c937-423a-ac02-df6e2dcd0bb2 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:00.674531 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da513b-c937-423a-ac02-df6e2dcd0bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:00.674893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da513b-c937-423a-ac02-df6e2dcd0bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:00.692678 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:00.770836 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-e6da513b-c937-423a-ac02-df6e2dcd0bb2 None None] MFA Rules not processed for user `c4a3579d56294b9d8ad36d4d174c8526`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:00.772548 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e6da513b-c937-423a-ac02-df6e2dcd0bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:00.905065 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d636d-1f82-4b61-be39-5dcfdb8cce93 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:01:00.906984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-bd9d636d-1f82-4b61-be39-5dcfdb8cce93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:00.915599 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 522/1036] 10.222.0.13 () {58 vars in 1058 bytes} [Sun Oct 23 00:01:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:01:01.058209 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d636d-1f82-4b61-be39-5dcfdb8cce93 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:01:01.140751 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:01.185896 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05648ba-65c8-4aba-b8fb-ffaf5ca949cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:01:01.186554 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05648ba-65c8-4aba-b8fb-ffaf5ca949cb service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:01.186822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05648ba-65c8-4aba-b8fb-ffaf5ca949cb service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:01.187012 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05648ba-65c8-4aba-b8fb-ffaf5ca949cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:01.189098 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b05648ba-65c8-4aba-b8fb-ffaf5ca949cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:01.290856 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05648ba-65c8-4aba-b8fb-ffaf5ca949cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:01:01.292622 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05648ba-65c8-4aba-b8fb-ffaf5ca949cb service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:01:01.314082 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 515/1037] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:01:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:01:03.713533 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fe91a-97b7-465b-8c93-4a28f0ee8084 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:03.714193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fe91a-97b7-465b-8c93-4a28f0ee8084 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:03.714193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fe91a-97b7-465b-8c93-4a28f0ee8084 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:03.715647 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 523/1038] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:01:03.721879 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cac331-33a2-45e2-89e0-369a72a98801 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:03.722413 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cac331-33a2-45e2-89e0-369a72a98801 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:03.722826 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cac331-33a2-45e2-89e0-369a72a98801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:03.797339 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-b7cac331-33a2-45e2-89e0-369a72a98801 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:03.798643 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b7cac331-33a2-45e2-89e0-369a72a98801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:03.838152 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:03.901324 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 516/1039] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:01:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:01:04.192458 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588fe91a-97b7-465b-8c93-4a28f0ee8084 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:01:04.194017 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-588fe91a-97b7-465b-8c93-4a28f0ee8084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:04.265691 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:04.375460 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588fe91a-97b7-465b-8c93-4a28f0ee8084 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:01:04.506729 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2d651-42c7-416f-bd9f-0dc251b05fd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:01:04.507340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2d651-42c7-416f-bd9f-0dc251b05fd3 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:04.507619 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2d651-42c7-416f-bd9f-0dc251b05fd3 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:04.507907 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2d651-42c7-416f-bd9f-0dc251b05fd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:04.509503 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f3a2d651-42c7-416f-bd9f-0dc251b05fd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:04.532810 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:04.605904 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2d651-42c7-416f-bd9f-0dc251b05fd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:01:04.607635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2d651-42c7-416f-bd9f-0dc251b05fd3 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:01:04.626699 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 524/1040] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:01:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:01:07.133608 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085681e0-d4f9-4e3b-b04e-80afe24db305 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:07.134166 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085681e0-d4f9-4e3b-b04e-80afe24db305 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:07.134166 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085681e0-d4f9-4e3b-b04e-80afe24db305 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:07.135511 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 517/1041] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:01:07.140874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc2d76-b679-4eef-9f47-7100688cc674 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:07.141259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc2d76-b679-4eef-9f47-7100688cc674 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:07.141439 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbc2d76-b679-4eef-9f47-7100688cc674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:07.211088 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1dbc2d76-b679-4eef-9f47-7100688cc674 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:07.212383 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1dbc2d76-b679-4eef-9f47-7100688cc674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:07.317601 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:07.324763 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 525/1042] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:01:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:01:07.613949 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73ebaf-c1ad-4b75-b383-180461043fc6 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:07.614241 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73ebaf-c1ad-4b75-b383-180461043fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:07.614517 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73ebaf-c1ad-4b75-b383-180461043fc6 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:07.616017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 518/1043] 10.222.0.13 () {54 vars in 961 bytes} [Sun Oct 23 00:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:01:07.621305 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32d386-0a77-40c0-8eb7-991f9993f3b6 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:07.621684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32d386-0a77-40c0-8eb7-991f9993f3b6 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:07.622033 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32d386-0a77-40c0-8eb7-991f9993f3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:07.704094 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1c32d386-0a77-40c0-8eb7-991f9993f3b6 None None] MFA Rules not processed for user `c4a3579d56294b9d8ad36d4d174c8526`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:07.705336 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1c32d386-0a77-40c0-8eb7-991f9993f3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:07.827867 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 526/1044] 10.222.0.13 () {58 vars in 1058 bytes} [Sun Oct 23 00:01:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:01:07.837204 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73ebaf-c1ad-4b75-b383-180461043fc6 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:01:07.838791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-fd73ebaf-c1ad-4b75-b383-180461043fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:07.993494 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73ebaf-c1ad-4b75-b383-180461043fc6 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:01:08.014418 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:08.133566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b6d82-81c1-4b21-949d-b6e46d1247a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:01:08.134334 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b6d82-81c1-4b21-949d-b6e46d1247a1 service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:08.134876 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b6d82-81c1-4b21-949d-b6e46d1247a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:08.135054 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b6d82-81c1-4b21-949d-b6e46d1247a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:08.136773 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-917b6d82-81c1-4b21-949d-b6e46d1247a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:08.242919 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b6d82-81c1-4b21-949d-b6e46d1247a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:01:08.244580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b6d82-81c1-4b21-949d-b6e46d1247a1 service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:01:08.263999 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 519/1045] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:01:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:01:08.314993 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b49cd-2c59-4f94-aeeb-2a6264cd15e6 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:08.316026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b49cd-2c59-4f94-aeeb-2a6264cd15e6 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:08.316116 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b49cd-2c59-4f94-aeeb-2a6264cd15e6 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:08.317609 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 527/1046] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:01:08.322924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a49a7a-ea1f-42cc-9b30-3a4640bc02d7 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:08.323220 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a49a7a-ea1f-42cc-9b30-3a4640bc02d7 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:08.323506 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a49a7a-ea1f-42cc-9b30-3a4640bc02d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:08.358601 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:08.411875 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-90a49a7a-ea1f-42cc-9b30-3a4640bc02d7 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:08.413270 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-90a49a7a-ea1f-42cc-9b30-3a4640bc02d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:08.532107 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 520/1047] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:01:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:01:08.864283 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b94e16-8121-463a-b4bc-dc84e4510e1c None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:08.864283 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b94e16-8121-463a-b4bc-dc84e4510e1c None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:08.864283 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b94e16-8121-463a-b4bc-dc84e4510e1c None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:08.866795 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 528/1048] 10.222.0.13 () {54 vars in 963 bytes} [Sun Oct 23 00:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:01:08.870304 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692baafe-c7d8-4d47-83ac-6ac121518860 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:08.870539 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692baafe-c7d8-4d47-83ac-6ac121518860 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:08.870747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692baafe-c7d8-4d47-83ac-6ac121518860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:08.950661 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-692baafe-c7d8-4d47-83ac-6ac121518860 None None] MFA Rules not processed for user `bd170de478a344fc98951697c7a6298d`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:08.951866 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-692baafe-c7d8-4d47-83ac-6ac121518860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:09.005608 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:09.039272 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 521/1049] 10.222.0.13 () {58 vars in 1060 bytes} [Sun Oct 23 00:01:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:01:09.046831 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b94e16-8121-463a-b4bc-dc84e4510e1c None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:01:09.048390 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-72b94e16-8121-463a-b4bc-dc84e4510e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:09.107680 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:09.165661 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b94e16-8121-463a-b4bc-dc84e4510e1c None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:01:09.253546 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f1af2-9c03-49dd-b65b-2f0995db1217 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:01:09.254183 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f1af2-9c03-49dd-b65b-2f0995db1217 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:09.254465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f1af2-9c03-49dd-b65b-2f0995db1217 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:09.254738 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f1af2-9c03-49dd-b65b-2f0995db1217 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:09.256478 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-0f4f1af2-9c03-49dd-b65b-2f0995db1217 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:09.388319 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:09.404572 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f1af2-9c03-49dd-b65b-2f0995db1217 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:01:09.405996 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f1af2-9c03-49dd-b65b-2f0995db1217 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:01:09.425895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 529/1050] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:01:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:01:10.535405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d696e-d8ce-4c4e-8cfe-06cbeab07670 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:10.535977 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d696e-d8ce-4c4e-8cfe-06cbeab07670 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:10.535977 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d696e-d8ce-4c4e-8cfe-06cbeab07670 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:10.537337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 522/1051] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:01:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:01:10.542567 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213265e3-1e1d-412d-83ca-10e045128246 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:10.542973 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213265e3-1e1d-412d-83ca-10e045128246 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:10.543257 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213265e3-1e1d-412d-83ca-10e045128246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:10.608800 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-213265e3-1e1d-412d-83ca-10e045128246 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:10.609996 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-213265e3-1e1d-412d-83ca-10e045128246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:10.716214 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 530/1052] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:01:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:01:11.005427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d696e-d8ce-4c4e-8cfe-06cbeab07670 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:01:11.007021 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ad1d696e-d8ce-4c4e-8cfe-06cbeab07670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:11.161215 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d696e-d8ce-4c4e-8cfe-06cbeab07670 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:01:11.214787 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:11.281805 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75388de6-5065-4a73-b0a6-1a7c59df8277 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:01:11.282419 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75388de6-5065-4a73-b0a6-1a7c59df8277 service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:11.282703 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75388de6-5065-4a73-b0a6-1a7c59df8277 service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:11.282987 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75388de6-5065-4a73-b0a6-1a7c59df8277 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:11.284725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-75388de6-5065-4a73-b0a6-1a7c59df8277 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:11.379807 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75388de6-5065-4a73-b0a6-1a7c59df8277 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:01:11.381512 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75388de6-5065-4a73-b0a6-1a7c59df8277 service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:01:11.399903 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 523/1053] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:01:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:01:13.619232 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443b734-4a9d-43d1-bb8a-8b6c8c8948ac None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:13.619871 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443b734-4a9d-43d1-bb8a-8b6c8c8948ac None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:13.620078 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443b734-4a9d-43d1-bb8a-8b6c8c8948ac None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:13.621883 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 531/1054] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:01:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:01:13.627170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4123f8-48cf-483a-a3cd-245fa41059b6 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:13.627607 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4123f8-48cf-483a-a3cd-245fa41059b6 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:13.627704 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4123f8-48cf-483a-a3cd-245fa41059b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:13.702859 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-bb4123f8-48cf-483a-a3cd-245fa41059b6 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:13.704296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-bb4123f8-48cf-483a-a3cd-245fa41059b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:13.712165 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:13.817030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 524/1055] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:01:14.109056 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443b734-4a9d-43d1-bb8a-8b6c8c8948ac None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:01:14.110591 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-3443b734-4a9d-43d1-bb8a-8b6c8c8948ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:14.176461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:14.269100 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443b734-4a9d-43d1-bb8a-8b6c8c8948ac None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:01:14.386667 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde20cb-1340-4833-b386-4fe263856945 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:01:14.387350 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde20cb-1340-4833-b386-4fe263856945 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:14.387742 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde20cb-1340-4833-b386-4fe263856945 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:14.388148 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde20cb-1340-4833-b386-4fe263856945 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:14.389860 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-edde20cb-1340-4833-b386-4fe263856945 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:14.411119 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:14.487717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edde20cb-1340-4833-b386-4fe263856945 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:01:14.489520 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edde20cb-1340-4833-b386-4fe263856945 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:01:14.510683 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 532/1056] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:01:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:01:16.758146 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0893d9-f5a4-48db-b385-24ca0ab4c8d6 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:16.758634 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0893d9-f5a4-48db-b385-24ca0ab4c8d6 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:16.758744 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0893d9-f5a4-48db-b385-24ca0ab4c8d6 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:16.760029 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 525/1057] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:01:16.765470 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98189d2c-0904-4148-bbfe-1ecc8bef43b4 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:16.765906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98189d2c-0904-4148-bbfe-1ecc8bef43b4 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:16.766125 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98189d2c-0904-4148-bbfe-1ecc8bef43b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:16.837520 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-98189d2c-0904-4148-bbfe-1ecc8bef43b4 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:16.838704 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-98189d2c-0904-4148-bbfe-1ecc8bef43b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:16.944723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:16.960871 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 533/1058] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:01:17.299465 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0893d9-f5a4-48db-b385-24ca0ab4c8d6 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:01:17.302189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-da0893d9-f5a4-48db-b385-24ca0ab4c8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:17.462808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:17.474280 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0893d9-f5a4-48db-b385-24ca0ab4c8d6 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:01:17.594604 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d485eac5-7d0d-43ea-9d48-1d2c3d049e2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:01:17.595312 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485eac5-7d0d-43ea-9d48-1d2c3d049e2f service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:17.595655 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485eac5-7d0d-43ea-9d48-1d2c3d049e2f service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:17.595973 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d485eac5-7d0d-43ea-9d48-1d2c3d049e2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:17.597832 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d485eac5-7d0d-43ea-9d48-1d2c3d049e2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:17.700799 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485eac5-7d0d-43ea-9d48-1d2c3d049e2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:01:17.702558 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d485eac5-7d0d-43ea-9d48-1d2c3d049e2f service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:01:17.714433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:17.723645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 526/1059] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:01:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:01:20.070415 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af801516-43b1-41b4-9358-fa389b9fcd79 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:20.071095 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af801516-43b1-41b4-9358-fa389b9fcd79 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:20.071095 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af801516-43b1-41b4-9358-fa389b9fcd79 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:20.072175 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 534/1060] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:01:20.077964 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56dcb0e-0f59-4e82-9db9-8c21173f9898 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:20.078713 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56dcb0e-0f59-4e82-9db9-8c21173f9898 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:20.078846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56dcb0e-0f59-4e82-9db9-8c21173f9898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:20.164202 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-c56dcb0e-0f59-4e82-9db9-8c21173f9898 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:20.165997 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c56dcb0e-0f59-4e82-9db9-8c21173f9898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:20.282926 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 527/1061] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:01:20.601600 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af801516-43b1-41b4-9358-fa389b9fcd79 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:01:20.603303 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-af801516-43b1-41b4-9358-fa389b9fcd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:20.766403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af801516-43b1-41b4-9358-fa389b9fcd79 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:01:20.839174 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:20.895543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab617a47-4051-480c-97e0-360d277eb2c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:01:20.896188 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab617a47-4051-480c-97e0-360d277eb2c0 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:20.896506 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab617a47-4051-480c-97e0-360d277eb2c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:20.896794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab617a47-4051-480c-97e0-360d277eb2c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:20.898673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ab617a47-4051-480c-97e0-360d277eb2c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:20.996873 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab617a47-4051-480c-97e0-360d277eb2c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:01:20.998455 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab617a47-4051-480c-97e0-360d277eb2c0 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:01:21.016567 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 535/1062] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:01:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:01:23.270003 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933e78c-3fd7-41e1-b556-756bcb5296ff None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:23.270587 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933e78c-3fd7-41e1-b556-756bcb5296ff None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:23.270587 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3933e78c-3fd7-41e1-b556-756bcb5296ff None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:23.271907 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 528/1063] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:01:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:01:23.277368 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbca41-a36b-4b66-a19b-0e4aa7c8ee92 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:23.277830 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbca41-a36b-4b66-a19b-0e4aa7c8ee92 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:23.277936 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbca41-a36b-4b66-a19b-0e4aa7c8ee92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:23.345843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-5ecbca41-a36b-4b66-a19b-0e4aa7c8ee92 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:23.347187 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5ecbca41-a36b-4b66-a19b-0e4aa7c8ee92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:23.381028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:23.445487 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 536/1064] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:01:23.736861 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933e78c-3fd7-41e1-b556-756bcb5296ff None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:01:23.738915 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-3933e78c-3fd7-41e1-b556-756bcb5296ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:23.817878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:23.887170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3933e78c-3fd7-41e1-b556-756bcb5296ff None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:01:23.998393 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce34c9-e705-4b55-b71e-211e8cb63c28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:01:23.998927 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce34c9-e705-4b55-b71e-211e8cb63c28 service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:23.999204 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce34c9-e705-4b55-b71e-211e8cb63c28 service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:23.999475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce34c9-e705-4b55-b71e-211e8cb63c28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:24.001083 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-7cce34c9-e705-4b55-b71e-211e8cb63c28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:24.039744 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:24.090438 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce34c9-e705-4b55-b71e-211e8cb63c28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:01:24.092016 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce34c9-e705-4b55-b71e-211e8cb63c28 service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:01:24.116162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 529/1065] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:01:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:01:24.740111 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79023c-ebf1-4f2e-bdad-616849e59f80 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:24.740736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79023c-ebf1-4f2e-bdad-616849e59f80 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:24.740736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79023c-ebf1-4f2e-bdad-616849e59f80 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:24.742034 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 537/1066] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:01:24.747157 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a4bd2-09e9-47d0-b04b-a35ca508214d None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:24.747581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a4bd2-09e9-47d0-b04b-a35ca508214d None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:24.747706 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a4bd2-09e9-47d0-b04b-a35ca508214d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:24.814151 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-a28a4bd2-09e9-47d0-b04b-a35ca508214d None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:24.815871 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a28a4bd2-09e9-47d0-b04b-a35ca508214d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:24.914667 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 530/1067] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:01:25.224570 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cee330-bb3c-44c4-af63-55e9248d6d1c None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:25.224930 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cee330-bb3c-44c4-af63-55e9248d6d1c None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:25.225274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cee330-bb3c-44c4-af63-55e9248d6d1c None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:25.226602 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 538/1068] 10.222.0.13 () {54 vars in 962 bytes} [Sun Oct 23 00:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:01:25.231958 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25aeeb-51ae-45a9-a2b5-61a57fb17766 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:25.232247 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25aeeb-51ae-45a9-a2b5-61a57fb17766 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:25.232547 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25aeeb-51ae-45a9-a2b5-61a57fb17766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:25.250131 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:25.311573 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-2d25aeeb-51ae-45a9-a2b5-61a57fb17766 None None] MFA Rules not processed for user `e4a69aeb3fba49b6881f1f15c1758494`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:25.312806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2d25aeeb-51ae-45a9-a2b5-61a57fb17766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:25.388628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 531/1069] 10.222.0.13 () {58 vars in 1059 bytes} [Sun Oct 23 00:01:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:01:25.396512 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cee330-bb3c-44c4-af63-55e9248d6d1c None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:01:25.398153 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-57cee330-bb3c-44c4-af63-55e9248d6d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:25.502135 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cee330-bb3c-44c4-af63-55e9248d6d1c None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:01:25.561518 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:25.577827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fcdcf1-5085-4e7d-be79-dc20371f3f5b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a69aeb3fba49b6881f1f15c1758494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:01:25.578434 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcdcf1-5085-4e7d-be79-dc20371f3f5b service glance] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:25.578981 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcdcf1-5085-4e7d-be79-dc20371f3f5b service glance] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:25.579260 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcdcf1-5085-4e7d-be79-dc20371f3f5b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:25.580892 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-08fcdcf1-5085-4e7d-be79-dc20371f3f5b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:25.692099 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fcdcf1-5085-4e7d-be79-dc20371f3f5b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:01:25.693519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fcdcf1-5085-4e7d-be79-dc20371f3f5b service glance] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:01:25.712741 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 539/1070] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:01:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:01:26.328869 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e5424-a3cb-4b38-a741-7a81c821966b None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:26.329433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e5424-a3cb-4b38-a741-7a81c821966b None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:26.329433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e5424-a3cb-4b38-a741-7a81c821966b None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:26.330773 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 532/1071] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:01:26.337155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb32e0-a3f3-44c1-9ea1-5bebad3869be None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:26.337575 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb32e0-a3f3-44c1-9ea1-5bebad3869be None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:26.337708 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cb32e0-a3f3-44c1-9ea1-5bebad3869be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:26.405358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-f6cb32e0-a3f3-44c1-9ea1-5bebad3869be None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:26.407004 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f6cb32e0-a3f3-44c1-9ea1-5bebad3869be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:26.451201 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:26.455201 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db34aae-5566-45c0-aed5-79ee92f6c050 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:26.455502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db34aae-5566-45c0-aed5-79ee92f6c050 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:26.455813 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db34aae-5566-45c0-aed5-79ee92f6c050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:26.512596 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 540/1072] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:01:26.524792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-8db34aae-5566-45c0-aed5-79ee92f6c050 None None] MFA Rules not processed for user `7c0356072f92451f98b03a2225e22f86`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:26.526011 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8db34aae-5566-45c0-aed5-79ee92f6c050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:26.581197 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:26.615886 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 533/1073] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 00:01:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:01:26.662467 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1239ba-4cd4-4084-af89-b6c8cbff891c None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:26.662838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1239ba-4cd4-4084-af89-b6c8cbff891c None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:26.663189 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1239ba-4cd4-4084-af89-b6c8cbff891c None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:26.664820 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 541/1074] 10.222.0.13 () {54 vars in 943 bytes} [Sun Oct 23 00:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:01:26.672120 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130b50c-0517-44b0-a792-a1756446a69c None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:26.673128 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130b50c-0517-44b0-a792-a1756446a69c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:26.673128 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130b50c-0517-44b0-a792-a1756446a69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:26.761172 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-4130b50c-0517-44b0-a792-a1756446a69c None None] MFA Rules not processed for user `bfd8c7516b7c435d88298672b2596d6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:26.762396 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-4130b50c-0517-44b0-a792-a1756446a69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:26.803602 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1239ba-4cd4-4084-af89-b6c8cbff891c None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:01:26.805289 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-3a1239ba-4cd4-4084-af89-b6c8cbff891c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:26.876691 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 534/1075] 10.222.0.13 () {58 vars in 1040 bytes} [Sun Oct 23 00:01:26 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:01:26.886898 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130b50c-0517-44b0-a792-a1756446a69c None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:01:26.896862 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130b50c-0517-44b0-a792-a1756446a69c None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:01:26.922820 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:26.957211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1239ba-4cd4-4084-af89-b6c8cbff891c None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:01:26.983109 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:27.006786 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be20618e-8deb-4449-a788-8c0d012ebc2e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfd8c7516b7c435d88298672b2596d6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:01:27.007394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20618e-8deb-4449-a788-8c0d012ebc2e service swift] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:27.007671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20618e-8deb-4449-a788-8c0d012ebc2e service swift] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:27.007945 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be20618e-8deb-4449-a788-8c0d012ebc2e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:27.009643 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-be20618e-8deb-4449-a788-8c0d012ebc2e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:27.077021 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f441821-a3ce-42f6-b3d8-597ce7b9019a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:01:27.077871 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f441821-a3ce-42f6-b3d8-597ce7b9019a service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:27.078159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f441821-a3ce-42f6-b3d8-597ce7b9019a service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:27.078433 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f441821-a3ce-42f6-b3d8-597ce7b9019a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:27.080261 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-6f441821-a3ce-42f6-b3d8-597ce7b9019a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:27.096091 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be20618e-8deb-4449-a788-8c0d012ebc2e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:01:27.098772 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be20618e-8deb-4449-a788-8c0d012ebc2e service swift] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:01:27.107276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 535/1076] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 00:01:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 23 00:01:27.176101 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f441821-a3ce-42f6-b3d8-597ce7b9019a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:01:27.177716 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f441821-a3ce-42f6-b3d8-597ce7b9019a service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:01:27.197397 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 542/1077] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:01:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:01:29.488868 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669aed1-d068-477b-b121-c4421ae49915 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:29.489381 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669aed1-d068-477b-b121-c4421ae49915 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:29.489381 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669aed1-d068-477b-b121-c4421ae49915 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:29.490668 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 536/1078] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:01:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:01:29.496237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89fa95-6488-4641-99e6-1bb284372319 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:29.496736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89fa95-6488-4641-99e6-1bb284372319 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:29.496847 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89fa95-6488-4641-99e6-1bb284372319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:29.538090 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:29.572168 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-cb89fa95-6488-4641-99e6-1bb284372319 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:29.573454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-cb89fa95-6488-4641-99e6-1bb284372319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:29.693446 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 543/1079] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:01:29.991055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669aed1-d068-477b-b121-c4421ae49915 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:01:29.992628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a669aed1-d068-477b-b121-c4421ae49915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:30.066624 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:30.164511 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669aed1-d068-477b-b121-c4421ae49915 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:01:30.301893 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b0c7a9-fa97-40b5-8dcf-d04e6dbc3042 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:01:30.302543 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0c7a9-fa97-40b5-8dcf-d04e6dbc3042 service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:30.302826 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0c7a9-fa97-40b5-8dcf-d04e6dbc3042 service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:30.303105 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b0c7a9-fa97-40b5-8dcf-d04e6dbc3042 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:30.304734 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-76b0c7a9-fa97-40b5-8dcf-d04e6dbc3042 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:30.326016 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:30.406526 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b0c7a9-fa97-40b5-8dcf-d04e6dbc3042 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:01:30.408472 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b0c7a9-fa97-40b5-8dcf-d04e6dbc3042 service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:01:30.427550 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 537/1080] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:01:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:01:32.702303 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b31d1-3895-4306-8bb7-79308834edfa None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:32.703289 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b31d1-3895-4306-8bb7-79308834edfa None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:32.703289 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b31d1-3895-4306-8bb7-79308834edfa None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:32.704566 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 544/1081] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:01:32.710710 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fdbcb-e556-4182-b7c7-eb543cdfebb1 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:32.711156 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fdbcb-e556-4182-b7c7-eb543cdfebb1 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:32.711247 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fdbcb-e556-4182-b7c7-eb543cdfebb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:32.784232 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-714fdbcb-e556-4182-b7c7-eb543cdfebb1 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:32.785623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-714fdbcb-e556-4182-b7c7-eb543cdfebb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:32.885135 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:32.892341 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 538/1082] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:01:33.188312 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b31d1-3895-4306-8bb7-79308834edfa None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:01:33.189879 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-187b31d1-3895-4306-8bb7-79308834edfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:33.320676 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:33.362962 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b31d1-3895-4306-8bb7-79308834edfa None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:01:33.494523 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d69322-76f6-4be0-bf28-49959c65a60d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:01:33.495142 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d69322-76f6-4be0-bf28-49959c65a60d service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:33.495425 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d69322-76f6-4be0-bf28-49959c65a60d service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:33.495700 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d69322-76f6-4be0-bf28-49959c65a60d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:33.497342 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-75d69322-76f6-4be0-bf28-49959c65a60d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:33.598404 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:33.622929 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d69322-76f6-4be0-bf28-49959c65a60d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:01:33.624465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d69322-76f6-4be0-bf28-49959c65a60d service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:01:33.646560 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 545/1083] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:01:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:01:35.918427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8796b03-e05a-4e63-b3d9-e91721142174 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:35.919028 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8796b03-e05a-4e63-b3d9-e91721142174 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:35.919028 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8796b03-e05a-4e63-b3d9-e91721142174 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:35.920309 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 539/1084] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:01:35.925982 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add305e4-cb23-42ab-99b1-9d086c6d4fe0 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:35.926418 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add305e4-cb23-42ab-99b1-9d086c6d4fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:35.926516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add305e4-cb23-42ab-99b1-9d086c6d4fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:35.998717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-add305e4-cb23-42ab-99b1-9d086c6d4fe0 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:35.999947 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-add305e4-cb23-42ab-99b1-9d086c6d4fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:36.114402 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 546/1085] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:01:36.406966 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8796b03-e05a-4e63-b3d9-e91721142174 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:01:36.408530 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a8796b03-e05a-4e63-b3d9-e91721142174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:36.575364 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8796b03-e05a-4e63-b3d9-e91721142174 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:01:36.652063 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:36.709840 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261af1d-9bcd-491a-825c-fb7016ee37f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:01:36.710554 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261af1d-9bcd-491a-825c-fb7016ee37f9 service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:36.711169 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261af1d-9bcd-491a-825c-fb7016ee37f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:36.711599 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261af1d-9bcd-491a-825c-fb7016ee37f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:36.713702 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9261af1d-9bcd-491a-825c-fb7016ee37f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:36.817705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261af1d-9bcd-491a-825c-fb7016ee37f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:01:36.819289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261af1d-9bcd-491a-825c-fb7016ee37f9 service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:01:36.838269 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 540/1086] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:01:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:01:46.112278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=79 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:46.835815 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=64 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:50.017227 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab2751-7509-448f-b9d2-886e7abb9405 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:50.017802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab2751-7509-448f-b9d2-886e7abb9405 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:50.017802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab2751-7509-448f-b9d2-886e7abb9405 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:50.019231 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 547/1087] 10.222.0.13 () {54 vars in 918 bytes} [Sun Oct 23 00:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:01:50.024219 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cb488-eaa8-4bc6-964a-089067a7c998 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:50.024674 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cb488-eaa8-4bc6-964a-089067a7c998 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:50.024801 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cb488-eaa8-4bc6-964a-089067a7c998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:50.098316 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-6e1cb488-eaa8-4bc6-964a-089067a7c998 None None] MFA Rules not processed for user `24108b6355414f64ba5e7bf84011eee6`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:50.099512 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-6e1cb488-eaa8-4bc6-964a-089067a7c998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:50.205641 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 541/1088] 10.222.0.13 () {58 vars in 1015 bytes} [Sun Oct 23 00:01:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:01:50.219966 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40d1c4-f0d4-4014-b778-b17074d0239e None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:50.220254 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40d1c4-f0d4-4014-b778-b17074d0239e None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:50.220550 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40d1c4-f0d4-4014-b778-b17074d0239e None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:50.221905 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 548/1089] 10.222.0.13 () {54 vars in 918 bytes} [Sun Oct 23 00:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:01:50.226587 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc388eb7-5d2e-4e63-80a8-f59e656ff798 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:50.226872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc388eb7-5d2e-4e63-80a8-f59e656ff798 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:50.227147 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc388eb7-5d2e-4e63-80a8-f59e656ff798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:50.298905 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:50.302193 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-fc388eb7-5d2e-4e63-80a8-f59e656ff798 None None] MFA Rules not processed for user `c4a3579d56294b9d8ad36d4d174c8526`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:50.303996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-fc388eb7-5d2e-4e63-80a8-f59e656ff798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:50.420296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 542/1090] 10.222.0.13 () {58 vars in 1015 bytes} [Sun Oct 23 00:01:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:01:53.017138 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86f288-22c7-41a4-839c-ae7eae64283a None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:53.017713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86f288-22c7-41a4-839c-ae7eae64283a None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:53.017713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86f288-22c7-41a4-839c-ae7eae64283a None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:53.019635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 549/1091] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:01:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:01:53.025008 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72d7ca-36f8-4fcf-ba44-0f3ce5912606 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:53.025407 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72d7ca-36f8-4fcf-ba44-0f3ce5912606 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:53.025572 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72d7ca-36f8-4fcf-ba44-0f3ce5912606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:53.090624 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-ed72d7ca-36f8-4fcf-ba44-0f3ce5912606 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:53.091828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ed72d7ca-36f8-4fcf-ba44-0f3ce5912606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:53.147087 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:53.188307 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 543/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:01:53.486298 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86f288-22c7-41a4-839c-ae7eae64283a None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:01:53.488066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ae86f288-22c7-41a4-839c-ae7eae64283a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:53.634157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86f288-22c7-41a4-839c-ae7eae64283a None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:01:53.707206 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:53.747035 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90b93c-4f10-4f75-bb91-0120a5a35c6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:01:53.747565 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90b93c-4f10-4f75-bb91-0120a5a35c6b service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:53.747848 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90b93c-4f10-4f75-bb91-0120a5a35c6b service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:53.748125 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90b93c-4f10-4f75-bb91-0120a5a35c6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:53.749764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9b90b93c-4f10-4f75-bb91-0120a5a35c6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:53.839010 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90b93c-4f10-4f75-bb91-0120a5a35c6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:01:53.840603 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90b93c-4f10-4f75-bb91-0120a5a35c6b service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:01:53.858366 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 550/1093] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:01:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:01:54.995327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372aebe-c614-43a2-a153-ca3ba69ec897 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:54.995870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372aebe-c614-43a2-a153-ca3ba69ec897 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:54.995870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372aebe-c614-43a2-a153-ca3ba69ec897 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:54.997361 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 544/1094] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 00:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:01:55.002256 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63765d9e-272c-48af-89e2-74f52d7f2145 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:55.002670 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63765d9e-272c-48af-89e2-74f52d7f2145 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:55.002807 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63765d9e-272c-48af-89e2-74f52d7f2145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:55.080498 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-63765d9e-272c-48af-89e2-74f52d7f2145 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:55.082701 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-63765d9e-272c-48af-89e2-74f52d7f2145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:55.110122 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:55.181880 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 551/1095] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 00:01:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:01:55.330354 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372aebe-c614-43a2-a153-ca3ba69ec897 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:01:55.332741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8372aebe-c614-43a2-a153-ca3ba69ec897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:55.474698 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372aebe-c614-43a2-a153-ca3ba69ec897 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:01:55.515337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:55.589692 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de4344-5c52-4c2d-a162-ee973d33e70c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:01:55.590301 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de4344-5c52-4c2d-a162-ee973d33e70c service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:55.590583 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de4344-5c52-4c2d-a162-ee973d33e70c service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:55.590860 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de4344-5c52-4c2d-a162-ee973d33e70c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:55.592454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-36de4344-5c52-4c2d-a162-ee973d33e70c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:55.680277 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de4344-5c52-4c2d-a162-ee973d33e70c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:01:55.681892 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de4344-5c52-4c2d-a162-ee973d33e70c service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:01:55.699608 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 545/1096] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:01:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:01:56.823908 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbfb52-9297-4b7f-ac8a-d78fc92dabde None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:56.824449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbfb52-9297-4b7f-ac8a-d78fc92dabde None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:56.824449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbfb52-9297-4b7f-ac8a-d78fc92dabde None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:56.825759 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 552/1097] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 00:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:01:56.830968 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1ef74-7979-402b-8884-e41c5c2d4d11 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:56.831915 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1ef74-7979-402b-8884-e41c5c2d4d11 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:56.832678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1ef74-7979-402b-8884-e41c5c2d4d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:56.886496 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:56.909024 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-3dd1ef74-7979-402b-8884-e41c5c2d4d11 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:56.910331 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-3dd1ef74-7979-402b-8884-e41c5c2d4d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:57.006034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 546/1098] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 00:01:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:01:57.156398 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbfb52-9297-4b7f-ac8a-d78fc92dabde None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:01:57.157928 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-86cbfb52-9297-4b7f-ac8a-d78fc92dabde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:57.304370 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbfb52-9297-4b7f-ac8a-d78fc92dabde None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:01:57.316309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:57.415966 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed3317-0a69-4576-99ec-5eb278c8dbea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:01:57.416515 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed3317-0a69-4576-99ec-5eb278c8dbea service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:57.416799 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed3317-0a69-4576-99ec-5eb278c8dbea service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:57.417072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed3317-0a69-4576-99ec-5eb278c8dbea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:57.418645 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-65ed3317-0a69-4576-99ec-5eb278c8dbea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:57.506221 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed3317-0a69-4576-99ec-5eb278c8dbea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:01:57.507779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed3317-0a69-4576-99ec-5eb278c8dbea service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:01:57.525840 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 553/1099] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:01:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:01:58.624605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac455d-73c4-4643-acc9-1e50737d629b None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:58.625157 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac455d-73c4-4643-acc9-1e50737d629b None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:58.625157 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac455d-73c4-4643-acc9-1e50737d629b None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:58.626517 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 547/1100] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 00:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:01:58.632032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df116d6-b120-46d4-952c-c06f616033c0 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:58.632460 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df116d6-b120-46d4-952c-c06f616033c0 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:58.632631 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df116d6-b120-46d4-952c-c06f616033c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:58.652692 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:58.710408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-9df116d6-b120-46d4-952c-c06f616033c0 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:01:58.711617 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9df116d6-b120-46d4-952c-c06f616033c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:58.806302 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 554/1101] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 00:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:01:58.955321 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac455d-73c4-4643-acc9-1e50737d629b None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:01:58.956924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f6ac455d-73c4-4643-acc9-1e50737d629b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:59.064543 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:59.100962 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac455d-73c4-4643-acc9-1e50737d629b None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:01:59.212700 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ede7d0-9621-453f-893a-58861534fcae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:01:59.213225 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ede7d0-9621-453f-893a-58861534fcae service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:01:59.213501 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ede7d0-9621-453f-893a-58861534fcae service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:01:59.213772 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ede7d0-9621-453f-893a-58861534fcae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:01:59.215370 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-70ede7d0-9621-453f-893a-58861534fcae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:01:59.290179 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:01:59.304124 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ede7d0-9621-453f-893a-58861534fcae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:01:59.305734 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ede7d0-9621-453f-893a-58861534fcae service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:01:59.323844 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 548/1102] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:01:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:02:00.430732 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecac04-aacc-4c39-a5eb-bcbdb438bf6e None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:00.431259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecac04-aacc-4c39-a5eb-bcbdb438bf6e None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:00.431259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecac04-aacc-4c39-a5eb-bcbdb438bf6e None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:00.432603 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 555/1103] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 00:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:02:00.437670 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f34a68-ee4e-4318-a236-4e472f69f31c None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:00.438092 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f34a68-ee4e-4318-a236-4e472f69f31c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:00.438230 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f34a68-ee4e-4318-a236-4e472f69f31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:00.512799 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-d8f34a68-ee4e-4318-a236-4e472f69f31c None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:02:00.514023 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d8f34a68-ee4e-4318-a236-4e472f69f31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:00.614347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 549/1104] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 00:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:02:00.766664 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ecac04-aacc-4c39-a5eb-bcbdb438bf6e None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:02:00.768143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d4ecac04-aacc-4c39-a5eb-bcbdb438bf6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:00.851810 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:00.913397 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ecac04-aacc-4c39-a5eb-bcbdb438bf6e None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:02:01.025756 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd399d11-a481-4353-8ba0-171424533271 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:02:01.026277 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd399d11-a481-4353-8ba0-171424533271 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:01.026556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd399d11-a481-4353-8ba0-171424533271 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:01.026826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd399d11-a481-4353-8ba0-171424533271 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:01.028382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-bd399d11-a481-4353-8ba0-171424533271 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:01.076868 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:01.120076 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd399d11-a481-4353-8ba0-171424533271 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:02:01.121692 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd399d11-a481-4353-8ba0-171424533271 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:02:01.140335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 556/1105] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:02:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:02:02.185205 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d628fe-8ca5-46f3-876b-4cc4b54d9c18 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:02.185777 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d628fe-8ca5-46f3-876b-4cc4b54d9c18 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:02.185777 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d628fe-8ca5-46f3-876b-4cc4b54d9c18 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:02.187128 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 550/1106] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 00:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:02:02.192141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f021f1a-c663-4e4e-b3d5-88dcfc138da8 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:02.192685 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f021f1a-c663-4e4e-b3d5-88dcfc138da8 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:02.192793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f021f1a-c663-4e4e-b3d5-88dcfc138da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:02.269985 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-5f021f1a-c663-4e4e-b3d5-88dcfc138da8 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:02:02.271111 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5f021f1a-c663-4e4e-b3d5-88dcfc138da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:02.367184 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 557/1107] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 00:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:02:02.526664 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d628fe-8ca5-46f3-876b-4cc4b54d9c18 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:02:02.528128 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-91d628fe-8ca5-46f3-876b-4cc4b54d9c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:02.576620 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:02.672342 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d628fe-8ca5-46f3-876b-4cc4b54d9c18 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:02:02.783909 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ee05a-671c-4e13-8789-c19187a7f0c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:02:02.784753 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ee05a-671c-4e13-8789-c19187a7f0c1 service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:02.784929 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ee05a-671c-4e13-8789-c19187a7f0c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:02.785078 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ee05a-671c-4e13-8789-c19187a7f0c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:02.786656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-057ee05a-671c-4e13-8789-c19187a7f0c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:02.793451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:02.874486 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ee05a-671c-4e13-8789-c19187a7f0c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:02:02.876052 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ee05a-671c-4e13-8789-c19187a7f0c1 service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:02:02.893364 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 551/1108] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:02:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:02:03.941415 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f45533-d339-4cf5-8921-76c0b59d4d60 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:03.941984 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f45533-d339-4cf5-8921-76c0b59d4d60 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:03.941984 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f45533-d339-4cf5-8921-76c0b59d4d60 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:03.943313 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 558/1109] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 00:02:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:02:03.948124 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9abcf-05a9-48cf-903e-51774df82a68 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:03.948572 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9abcf-05a9-48cf-903e-51774df82a68 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:03.948786 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9abcf-05a9-48cf-903e-51774df82a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:04.025118 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-f2d9abcf-05a9-48cf-903e-51774df82a68 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:02:04.026303 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f2d9abcf-05a9-48cf-903e-51774df82a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:04.094132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:04.123529 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 552/1110] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 00:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:02:04.275344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f45533-d339-4cf5-8921-76c0b59d4d60 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:02:04.276955 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-c1f45533-d339-4cf5-8921-76c0b59d4d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:04.295004 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:04.422904 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f45533-d339-4cf5-8921-76c0b59d4d60 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:02:04.513515 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:04.540339 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be547c2f-1325-4b55-906a-52ac6dfb6864 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:02:04.541183 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be547c2f-1325-4b55-906a-52ac6dfb6864 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:04.541462 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be547c2f-1325-4b55-906a-52ac6dfb6864 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:04.541735 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be547c2f-1325-4b55-906a-52ac6dfb6864 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:04.543304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-be547c2f-1325-4b55-906a-52ac6dfb6864 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:04.633026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be547c2f-1325-4b55-906a-52ac6dfb6864 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:02:04.634605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be547c2f-1325-4b55-906a-52ac6dfb6864 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:02:04.652593 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 559/1111] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:02:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:02:05.694980 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32ec19-c2fa-4acd-953d-495ae7006ea6 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:05.695548 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32ec19-c2fa-4acd-953d-495ae7006ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:05.695548 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32ec19-c2fa-4acd-953d-495ae7006ea6 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:05.696979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 553/1112] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 00:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:02:05.702351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee03c4e-c3f8-4e78-b6c5-986bd35e3ff2 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:05.702768 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee03c4e-c3f8-4e78-b6c5-986bd35e3ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:05.702896 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee03c4e-c3f8-4e78-b6c5-986bd35e3ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:05.776982 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-7ee03c4e-c3f8-4e78-b6c5-986bd35e3ff2 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:02:05.778169 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-7ee03c4e-c3f8-4e78-b6c5-986bd35e3ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:05.818688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:05.871882 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 560/1113] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 00:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:02:06.020731 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32ec19-c2fa-4acd-953d-495ae7006ea6 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:02:06.022493 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9d32ec19-c2fa-4acd-953d-495ae7006ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:06.172073 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d32ec19-c2fa-4acd-953d-495ae7006ea6 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:02:06.227713 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:06.285929 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07234cf6-9dd1-40ab-acc4-504bb17571eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:02:06.286457 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07234cf6-9dd1-40ab-acc4-504bb17571eb service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:06.286739 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07234cf6-9dd1-40ab-acc4-504bb17571eb service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:06.287015 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07234cf6-9dd1-40ab-acc4-504bb17571eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:06.288643 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-07234cf6-9dd1-40ab-acc4-504bb17571eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:06.377497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07234cf6-9dd1-40ab-acc4-504bb17571eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:02:06.379107 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07234cf6-9dd1-40ab-acc4-504bb17571eb service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:02:06.396472 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 554/1114] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:02:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:02:07.441869 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f323e3-f751-423d-b240-269a32c9d297 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:07.442402 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f323e3-f751-423d-b240-269a32c9d297 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:07.442402 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f323e3-f751-423d-b240-269a32c9d297 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:07.444017 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 561/1115] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 00:02:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:02:07.449606 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a563176-4a00-493d-abe2-997716602700 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:07.450074 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a563176-4a00-493d-abe2-997716602700 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:07.450187 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a563176-4a00-493d-abe2-997716602700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:07.526273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-0a563176-4a00-493d-abe2-997716602700 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:02:07.527465 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-0a563176-4a00-493d-abe2-997716602700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:07.529506 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:07.624473 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 555/1116] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 00:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:02:07.777741 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f323e3-f751-423d-b240-269a32c9d297 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:02:07.779170 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a6f323e3-f751-423d-b240-269a32c9d297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:07.927948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f323e3-f751-423d-b240-269a32c9d297 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:02:07.954697 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:08.042337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b68d5-d69f-4f44-9455-e5dc298366bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:02:08.042859 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b68d5-d69f-4f44-9455-e5dc298366bd service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:08.043132 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b68d5-d69f-4f44-9455-e5dc298366bd service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:08.043413 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b68d5-d69f-4f44-9455-e5dc298366bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:08.044993 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-641b68d5-d69f-4f44-9455-e5dc298366bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:08.137237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b68d5-d69f-4f44-9455-e5dc298366bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:02:08.138814 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641b68d5-d69f-4f44-9455-e5dc298366bd service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:02:08.157025 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 562/1117] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:02:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:02:10.205412 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32704f7-8fa2-4719-b03d-ec8469ad5822 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:10.205984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32704f7-8fa2-4719-b03d-ec8469ad5822 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:10.206640 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32704f7-8fa2-4719-b03d-ec8469ad5822 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:10.208584 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 556/1118] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:02:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:02:10.214073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f8e56-1b81-4587-a9bf-7118b6d2f9db None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:10.214526 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f8e56-1b81-4587-a9bf-7118b6d2f9db None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:10.214705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f8e56-1b81-4587-a9bf-7118b6d2f9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:10.257952 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:10.286574 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-701f8e56-1b81-4587-a9bf-7118b6d2f9db None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:02:10.287879 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-701f8e56-1b81-4587-a9bf-7118b6d2f9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:10.387557 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 563/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:02:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:02:10.673892 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32704f7-8fa2-4719-b03d-ec8469ad5822 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:02:10.675248 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b32704f7-8fa2-4719-b03d-ec8469ad5822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:10.780339 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32704f7-8fa2-4719-b03d-ec8469ad5822 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:02:10.811717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:10.853513 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95fce53-6ab0-49d2-96bb-6e4838513d59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:02:10.854117 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fce53-6ab0-49d2-96bb-6e4838513d59 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:10.854397 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fce53-6ab0-49d2-96bb-6e4838513d59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:10.854671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fce53-6ab0-49d2-96bb-6e4838513d59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:10.856251 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d95fce53-6ab0-49d2-96bb-6e4838513d59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:10.964129 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95fce53-6ab0-49d2-96bb-6e4838513d59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:02:10.965497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95fce53-6ab0-49d2-96bb-6e4838513d59 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:02:10.982911 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 557/1120] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:02:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:02:13.276380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefae68-dc96-48b4-a634-6f00a6ccb452 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:13.276827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefae68-dc96-48b4-a634-6f00a6ccb452 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:13.276986 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefae68-dc96-48b4-a634-6f00a6ccb452 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:13.278393 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 564/1121] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:02:13.283660 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64438d-4bb8-4d5e-a40f-6a32db2ab276 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:13.284092 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64438d-4bb8-4d5e-a40f-6a32db2ab276 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:13.284235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64438d-4bb8-4d5e-a40f-6a32db2ab276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:13.350770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-7a64438d-4bb8-4d5e-a40f-6a32db2ab276 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:02:13.351981 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-7a64438d-4bb8-4d5e-a40f-6a32db2ab276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:13.364512 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:13.447742 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 558/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:02:13.733819 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefae68-dc96-48b4-a634-6f00a6ccb452 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:02:13.735353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-faefae68-dc96-48b4-a634-6f00a6ccb452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:13.839844 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefae68-dc96-48b4-a634-6f00a6ccb452 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:02:13.852662 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:13.914274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9926bd-08e1-4ad1-b659-396dcb0707e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:02:13.914907 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9926bd-08e1-4ad1-b659-396dcb0707e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:13.915190 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9926bd-08e1-4ad1-b659-396dcb0707e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:13.915467 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9926bd-08e1-4ad1-b659-396dcb0707e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:13.917059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9a9926bd-08e1-4ad1-b659-396dcb0707e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:14.024524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9926bd-08e1-4ad1-b659-396dcb0707e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:02:14.025868 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9926bd-08e1-4ad1-b659-396dcb0707e8 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:02:14.043412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 565/1123] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:02:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:02:16.484655 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa98568-f40a-40ac-af9f-185fc44ac3b2 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:16.485186 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa98568-f40a-40ac-af9f-185fc44ac3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:16.485186 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa98568-f40a-40ac-af9f-185fc44ac3b2 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:16.486470 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 559/1124] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:02:16.492474 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3926e-1258-4ae2-abf6-85333d2df830 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:16.492946 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3926e-1258-4ae2-abf6-85333d2df830 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:16.493058 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3926e-1258-4ae2-abf6-85333d2df830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:16.548394 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:16.567523 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-a5f3926e-1258-4ae2-abf6-85333d2df830 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:02:16.568766 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a5f3926e-1258-4ae2-abf6-85333d2df830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:16.665824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 566/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:02:16.953230 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa98568-f40a-40ac-af9f-185fc44ac3b2 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:02:16.954771 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8fa98568-f40a-40ac-af9f-185fc44ac3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:17.056849 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa98568-f40a-40ac-af9f-185fc44ac3b2 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:02:17.098005 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:17.132205 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe66813-5066-4d25-89ad-387575177f2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:02:17.132844 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe66813-5066-4d25-89ad-387575177f2d service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:17.133127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe66813-5066-4d25-89ad-387575177f2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:17.133401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe66813-5066-4d25-89ad-387575177f2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:17.134990 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-4fe66813-5066-4d25-89ad-387575177f2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:17.244333 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe66813-5066-4d25-89ad-387575177f2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:02:17.245797 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe66813-5066-4d25-89ad-387575177f2d service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:02:17.263712 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 560/1126] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:02:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:02:19.716696 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b7507-81b1-4321-8c71-50a64000bf5d None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:19.717262 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b7507-81b1-4321-8c71-50a64000bf5d None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:19.717262 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b7507-81b1-4321-8c71-50a64000bf5d None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:19.718595 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 567/1127] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:02:19.724459 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5dbec-c474-41ac-95d0-8d443cb5c39f None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:19.724913 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5dbec-c474-41ac-95d0-8d443cb5c39f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:19.725043 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5dbec-c474-41ac-95d0-8d443cb5c39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:19.790625 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-44e5dbec-c474-41ac-95d0-8d443cb5c39f None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:02:19.791831 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-44e5dbec-c474-41ac-95d0-8d443cb5c39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:19.812296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:19.888287 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 561/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:02:20.169643 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b7507-81b1-4321-8c71-50a64000bf5d None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:02:20.171222 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-682b7507-81b1-4321-8c71-50a64000bf5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:20.273793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b7507-81b1-4321-8c71-50a64000bf5d None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:02:20.294567 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:20.347839 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921c99d-1206-4ab5-81a5-ba1241cade6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:02:20.348435 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921c99d-1206-4ab5-81a5-ba1241cade6d service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:20.348748 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921c99d-1206-4ab5-81a5-ba1241cade6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:20.349024 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921c99d-1206-4ab5-81a5-ba1241cade6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:20.350605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-8921c99d-1206-4ab5-81a5-ba1241cade6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:20.458846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8921c99d-1206-4ab5-81a5-ba1241cade6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:02:20.460187 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8921c99d-1206-4ab5-81a5-ba1241cade6d service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:02:20.477979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 568/1129] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:02:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:02:22.761565 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f60c38-63c2-4e3c-a984-d1583191105d None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:22.762105 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f60c38-63c2-4e3c-a984-d1583191105d None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:22.762105 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f60c38-63c2-4e3c-a984-d1583191105d None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:22.763502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 562/1130] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:02:22.768835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4937f-add5-4db1-a912-33b639dfebf0 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:22.769295 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4937f-add5-4db1-a912-33b639dfebf0 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:22.769576 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4937f-add5-4db1-a912-33b639dfebf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:22.832626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:22.835780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1ad4937f-add5-4db1-a912-33b639dfebf0 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:02:22.837446 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1ad4937f-add5-4db1-a912-33b639dfebf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:22.933118 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 569/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:02:23.219289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f60c38-63c2-4e3c-a984-d1583191105d None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:02:23.220948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b4f60c38-63c2-4e3c-a984-d1583191105d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:23.323177 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f60c38-63c2-4e3c-a984-d1583191105d None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:02:23.372481 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:23.397330 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3535aa-61a6-41a3-ac3a-14a3652f4e1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:02:23.397926 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3535aa-61a6-41a3-ac3a-14a3652f4e1f service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:23.398206 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3535aa-61a6-41a3-ac3a-14a3652f4e1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:23.398478 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3535aa-61a6-41a3-ac3a-14a3652f4e1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:23.400055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8c3535aa-61a6-41a3-ac3a-14a3652f4e1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:23.507777 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3535aa-61a6-41a3-ac3a-14a3652f4e1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:02:23.509227 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3535aa-61a6-41a3-ac3a-14a3652f4e1f service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:02:23.527454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 563/1132] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:02:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:02:25.730281 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3f083-661a-428d-ae83-bc84daa31cf0 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:25.730822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3f083-661a-428d-ae83-bc84daa31cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:25.730822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3f083-661a-428d-ae83-bc84daa31cf0 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:25.732193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 570/1133] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:02:25.737851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71effd7f-03bb-48f3-87b6-986d2b957cf6 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:25.738339 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71effd7f-03bb-48f3-87b6-986d2b957cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:25.738657 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71effd7f-03bb-48f3-87b6-986d2b957cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:25.803345 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-71effd7f-03bb-48f3-87b6-986d2b957cf6 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:02:25.804575 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-71effd7f-03bb-48f3-87b6-986d2b957cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:25.833879 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:25.902731 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 564/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:02:26.191964 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3f083-661a-428d-ae83-bc84daa31cf0 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:02:26.193857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-eec3f083-661a-428d-ae83-bc84daa31cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:26.296708 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3f083-661a-428d-ae83-bc84daa31cf0 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:02:26.325612 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:26.373758 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841024a9-1057-47a3-8de2-e9dd7465a36f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:02:26.374317 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841024a9-1057-47a3-8de2-e9dd7465a36f service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:26.374601 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841024a9-1057-47a3-8de2-e9dd7465a36f service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:26.374877 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841024a9-1057-47a3-8de2-e9dd7465a36f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:26.376456 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-841024a9-1057-47a3-8de2-e9dd7465a36f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:26.485559 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841024a9-1057-47a3-8de2-e9dd7465a36f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:02:26.487004 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841024a9-1057-47a3-8de2-e9dd7465a36f service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:02:26.505404 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 571/1135] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:02:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:02:28.837522 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ea8ab-ecf7-4811-a90c-44e8a6113fa2 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:28.838065 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ea8ab-ecf7-4811-a90c-44e8a6113fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:28.838065 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ea8ab-ecf7-4811-a90c-44e8a6113fa2 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:28.839425 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 565/1136] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:02:28.844754 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5861e-83d4-4637-9266-d58d8a0cc78e None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:28.845215 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5861e-83d4-4637-9266-d58d8a0cc78e None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:28.845322 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5861e-83d4-4637-9266-d58d8a0cc78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:28.912390 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-6ea5861e-83d4-4637-9266-d58d8a0cc78e None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:02:28.913642 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-6ea5861e-83d4-4637-9266-d58d8a0cc78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:28.921889 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:29.008615 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 572/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:02:29.290757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466ea8ab-ecf7-4811-a90c-44e8a6113fa2 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:02:29.292297 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-466ea8ab-ecf7-4811-a90c-44e8a6113fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:29.394260 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466ea8ab-ecf7-4811-a90c-44e8a6113fa2 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:02:29.449467 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:29.468223 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4b65e-294b-427e-ac04-16ad5bb53fc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:02:29.468861 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4b65e-294b-427e-ac04-16ad5bb53fc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:29.469140 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4b65e-294b-427e-ac04-16ad5bb53fc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:29.469413 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4b65e-294b-427e-ac04-16ad5bb53fc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:29.470991 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9fa4b65e-294b-427e-ac04-16ad5bb53fc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:29.579111 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4b65e-294b-427e-ac04-16ad5bb53fc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:02:29.580823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4b65e-294b-427e-ac04-16ad5bb53fc8 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:02:29.599159 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 566/1138] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:02:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:02:31.801520 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86952ed5-827b-4cc4-ad03-bb4f1b722b8b None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:31.802099 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86952ed5-827b-4cc4-ad03-bb4f1b722b8b None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:31.802099 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86952ed5-827b-4cc4-ad03-bb4f1b722b8b None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:31.803727 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 573/1139] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:02:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:02:31.808959 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb6332-9d16-4356-907d-e2432d45215a None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:31.809388 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb6332-9d16-4356-907d-e2432d45215a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:31.809583 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb6332-9d16-4356-907d-e2432d45215a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:31.874086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-daeb6332-9d16-4356-907d-e2432d45215a None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:02:31.875270 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-daeb6332-9d16-4356-907d-e2432d45215a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:31.912689 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:31.970007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 567/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:02:32.251275 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86952ed5-827b-4cc4-ad03-bb4f1b722b8b None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:02:32.252848 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-86952ed5-827b-4cc4-ad03-bb4f1b722b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:32.354826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86952ed5-827b-4cc4-ad03-bb4f1b722b8b None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:02:32.391660 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:32.426779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90168112-173f-4d83-ab90-2efa743d21b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:02:32.427379 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90168112-173f-4d83-ab90-2efa743d21b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:32.427661 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90168112-173f-4d83-ab90-2efa743d21b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:32.427932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90168112-173f-4d83-ab90-2efa743d21b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:32.429529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-90168112-173f-4d83-ab90-2efa743d21b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:32.536267 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90168112-173f-4d83-ab90-2efa743d21b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:02:32.538439 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90168112-173f-4d83-ab90-2efa743d21b3 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:02:32.564319 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 574/1141] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:02:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:02:34.812855 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7341e7e-de04-415a-82ae-ac4fdc5b7105 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:34.813399 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7341e7e-de04-415a-82ae-ac4fdc5b7105 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:34.813399 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7341e7e-de04-415a-82ae-ac4fdc5b7105 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:34.814772 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 568/1142] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:02:34.820349 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707032e-5a3f-4e7f-89b7-4a436c8fb441 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:34.820820 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707032e-5a3f-4e7f-89b7-4a436c8fb441 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:34.820927 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707032e-5a3f-4e7f-89b7-4a436c8fb441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:34.886939 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-2707032e-5a3f-4e7f-89b7-4a436c8fb441 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:02:34.888125 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-2707032e-5a3f-4e7f-89b7-4a436c8fb441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:34.905825 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:34.985470 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 575/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:02:35.279019 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7341e7e-de04-415a-82ae-ac4fdc5b7105 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:02:35.280638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a7341e7e-de04-415a-82ae-ac4fdc5b7105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:35.392150 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7341e7e-de04-415a-82ae-ac4fdc5b7105 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:02:35.458836 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:35.465691 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0695e1-8834-4f47-abe4-7d242126ef13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:02:35.466300 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0695e1-8834-4f47-abe4-7d242126ef13 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:35.466583 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0695e1-8834-4f47-abe4-7d242126ef13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:35.466861 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0695e1-8834-4f47-abe4-7d242126ef13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:35.468591 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-0c0695e1-8834-4f47-abe4-7d242126ef13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:35.578989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0695e1-8834-4f47-abe4-7d242126ef13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:02:35.580416 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0695e1-8834-4f47-abe4-7d242126ef13 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:02:35.599402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 569/1144] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:02:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:02:37.811277 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016e7e1-f17c-4824-a571-c3f511b69aef None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:37.811847 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016e7e1-f17c-4824-a571-c3f511b69aef None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:37.811847 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016e7e1-f17c-4824-a571-c3f511b69aef None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:37.813185 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 576/1145] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:02:37.819128 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1c7c7-372c-4fe9-8030-185001c107a7 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:37.819553 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1c7c7-372c-4fe9-8030-185001c107a7 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:37.819677 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1c7c7-372c-4fe9-8030-185001c107a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:37.886272 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-97f1c7c7-372c-4fe9-8030-185001c107a7 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:02:37.887497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-97f1c7c7-372c-4fe9-8030-185001c107a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:37.936305 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:38.000471 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 570/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:02:38.292757 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016e7e1-f17c-4824-a571-c3f511b69aef None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:02:38.294358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5016e7e1-f17c-4824-a571-c3f511b69aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:38.397167 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016e7e1-f17c-4824-a571-c3f511b69aef None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:02:38.445126 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:38.471678 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2259382e-308c-4ab4-ae80-923d173b4659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:02:38.472826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259382e-308c-4ab4-ae80-923d173b4659 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:38.473109 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259382e-308c-4ab4-ae80-923d173b4659 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:38.473392 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259382e-308c-4ab4-ae80-923d173b4659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:38.474981 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-2259382e-308c-4ab4-ae80-923d173b4659 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:38.583983 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2259382e-308c-4ab4-ae80-923d173b4659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:02:38.585441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2259382e-308c-4ab4-ae80-923d173b4659 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:02:38.603835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 577/1147] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:02:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:02:47.998376 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=22 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:48.601553 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=66 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:58.496296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f365f-e534-4740-ba19-1d27376e9698 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:58.496856 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f365f-e534-4740-ba19-1d27376e9698 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:58.496856 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f365f-e534-4740-ba19-1d27376e9698 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:58.498141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 571/1148] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:02:58.504207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb25b1-5377-4e20-a6d4-09902332f515 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:58.504656 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb25b1-5377-4e20-a6d4-09902332f515 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:58.504799 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb25b1-5377-4e20-a6d4-09902332f515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:58.571390 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-f5fb25b1-5377-4e20-a6d4-09902332f515 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:02:58.572641 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f5fb25b1-5377-4e20-a6d4-09902332f515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:58.667494 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 578/1149] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:02:58.970295 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be39ac-ccc7-40a0-8788-b4a4a9eb2aea None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:58.970584 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be39ac-ccc7-40a0-8788-b4a4a9eb2aea None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:58.970885 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be39ac-ccc7-40a0-8788-b4a4a9eb2aea None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:58.972875 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 572/1150] 10.222.0.13 () {54 vars in 962 bytes} [Sun Oct 23 00:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:02:58.978014 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140afade-ef74-4334-b9b2-89993cd773f3 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:58.978304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140afade-ef74-4334-b9b2-89993cd773f3 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:58.978581 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140afade-ef74-4334-b9b2-89993cd773f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:59.053054 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-140afade-ef74-4334-b9b2-89993cd773f3 None None] MFA Rules not processed for user `e4a69aeb3fba49b6881f1f15c1758494`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:02:59.054241 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-140afade-ef74-4334-b9b2-89993cd773f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:59.074273 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:59.133456 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 579/1151] 10.222.0.13 () {58 vars in 1059 bytes} [Sun Oct 23 00:02:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:02:59.141180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be39ac-ccc7-40a0-8788-b4a4a9eb2aea None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:02:59.142709 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-44be39ac-ccc7-40a0-8788-b4a4a9eb2aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:59.246941 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be39ac-ccc7-40a0-8788-b4a4a9eb2aea None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:02:59.321017 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f6d89e-85fd-489b-a197-40c848677ac2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a69aeb3fba49b6881f1f15c1758494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:02:59.321623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6d89e-85fd-489b-a197-40c848677ac2 service glance] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:02:59.321907 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6d89e-85fd-489b-a197-40c848677ac2 service glance] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:02:59.322182 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f6d89e-85fd-489b-a197-40c848677ac2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:02:59.323781 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a6f6d89e-85fd-489b-a197-40c848677ac2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:02:59.378438 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:02:59.433922 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f6d89e-85fd-489b-a197-40c848677ac2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:02:59.435340 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f6d89e-85fd-489b-a197-40c848677ac2 service glance] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:02:59.453410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 573/1152] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:02:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:03:01.695747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcc058-6455-497b-8a5f-c37fa3a04032 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:03:01.696316 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcc058-6455-497b-8a5f-c37fa3a04032 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:03:01.696316 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcc058-6455-497b-8a5f-c37fa3a04032 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:03:01.697656 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 580/1153] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:03:01.703207 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0535eb0-3b07-43cb-8062-745c53031b49 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:03:01.703702 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0535eb0-3b07-43cb-8062-745c53031b49 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:03:01.703831 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0535eb0-3b07-43cb-8062-745c53031b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:03:01.773586 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-f0535eb0-3b07-43cb-8062-745c53031b49 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:03:01.774800 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f0535eb0-3b07-43cb-8062-745c53031b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:03:01.870582 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 574/1154] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:03:02.144951 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcc058-6455-497b-8a5f-c37fa3a04032 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:03:02.146484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-c6bcc058-6455-497b-8a5f-c37fa3a04032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:03:02.270393 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcc058-6455-497b-8a5f-c37fa3a04032 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:03:02.320284 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:03:02.364975 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19bce7-f6c5-45ea-832d-b192da4475ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:03:02.365509 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19bce7-f6c5-45ea-832d-b192da4475ae admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:03:02.365791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19bce7-f6c5-45ea-832d-b192da4475ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:03:02.366068 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19bce7-f6c5-45ea-832d-b192da4475ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:03:02.366701 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19bce7-f6c5-45ea-832d-b192da4475ae admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:03:02.367800 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: WARNING py.warnings [None req-5e19bce7-f6c5-45ea-832d-b192da4475ae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:03:02.367800 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: warnings.warn(msg) Oct 23 00:03:02.367800 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: Oct 23 00:03:02.368968 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19bce7-f6c5-45ea-832d-b192da4475ae admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:03:02.389060 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 581/1155] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 00:03:02 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:03:04.563810 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e90836-2446-4a58-9625-1bac7c8c29c5 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:03:04.564344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e90836-2446-4a58-9625-1bac7c8c29c5 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:03:04.564344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e90836-2446-4a58-9625-1bac7c8c29c5 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:03:04.566184 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 575/1156] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:03:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:03:04.571695 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8ffe2-b562-44bc-8213-81f9b90016ab None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:03:04.572120 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8ffe2-b562-44bc-8213-81f9b90016ab None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:03:04.572267 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8ffe2-b562-44bc-8213-81f9b90016ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:03:04.637819 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-98f8ffe2-b562-44bc-8213-81f9b90016ab None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:03:04.639039 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-98f8ffe2-b562-44bc-8213-81f9b90016ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:03:04.735117 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 582/1157] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:03:05.030948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e90836-2446-4a58-9625-1bac7c8c29c5 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:03:05.032635 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d7e90836-2446-4a58-9625-1bac7c8c29c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:03:05.042707 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:03:05.182695 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e90836-2446-4a58-9625-1bac7c8c29c5 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:03:05.264657 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:03:05.297437 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f6b7b7-cc0a-4065-a68b-b498aa0fac8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:03:05.297972 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6b7b7-cc0a-4065-a68b-b498aa0fac8d service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:03:05.298253 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6b7b7-cc0a-4065-a68b-b498aa0fac8d service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:03:05.298527 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f6b7b7-cc0a-4065-a68b-b498aa0fac8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:03:05.300641 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d2f6b7b7-cc0a-4065-a68b-b498aa0fac8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:03:05.390848 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f6b7b7-cc0a-4065-a68b-b498aa0fac8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:03:05.392441 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f6b7b7-cc0a-4065-a68b-b498aa0fac8d service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:03:05.410833 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 576/1158] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:03:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:03:07.758976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aef57a-9f3c-4a7e-8670-32f06301d638 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:03:07.759547 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aef57a-9f3c-4a7e-8670-32f06301d638 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:03:07.759547 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aef57a-9f3c-4a7e-8670-32f06301d638 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:03:07.760895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 583/1159] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:03:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:03:07.766163 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62041f21-1f93-43e4-8e74-05ae0456acff None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:03:07.766550 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62041f21-1f93-43e4-8e74-05ae0456acff None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:03:07.766715 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62041f21-1f93-43e4-8e74-05ae0456acff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:03:07.834348 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-62041f21-1f93-43e4-8e74-05ae0456acff None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:03:07.835575 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-62041f21-1f93-43e4-8e74-05ae0456acff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:03:07.880035 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:03:07.929992 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 577/1160] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:03:08.216636 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aef57a-9f3c-4a7e-8670-32f06301d638 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:03:08.218162 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b1aef57a-9f3c-4a7e-8670-32f06301d638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:03:08.241759 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:03:08.320940 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aef57a-9f3c-4a7e-8670-32f06301d638 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:03:08.393134 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832d4a51-ad7c-4a86-ab43-07c6a6845b8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:03:08.393662 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d4a51-ad7c-4a86-ab43-07c6a6845b8b service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:03:08.394072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d4a51-ad7c-4a86-ab43-07c6a6845b8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:03:08.394432 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d4a51-ad7c-4a86-ab43-07c6a6845b8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:03:08.395994 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-832d4a51-ad7c-4a86-ab43-07c6a6845b8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:03:08.469376 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:03:08.504159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832d4a51-ad7c-4a86-ab43-07c6a6845b8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:03:08.505606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832d4a51-ad7c-4a86-ab43-07c6a6845b8b service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:03:08.523693 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 584/1161] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:03:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:03:10.768387 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221629a-5512-43d1-a300-39cda01a0042 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:03:10.769032 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221629a-5512-43d1-a300-39cda01a0042 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:03:10.769032 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221629a-5512-43d1-a300-39cda01a0042 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:03:10.771029 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 578/1162] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:03:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:03:10.776570 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98700486-aab0-41ec-8a28-8b65573c79f5 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:03:10.776965 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98700486-aab0-41ec-8a28-8b65573c79f5 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:03:10.777140 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98700486-aab0-41ec-8a28-8b65573c79f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:03:10.843245 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-98700486-aab0-41ec-8a28-8b65573c79f5 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:03:10.844466 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-98700486-aab0-41ec-8a28-8b65573c79f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:03:10.941413 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 585/1163] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:03:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:03:11.235774 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221629a-5512-43d1-a300-39cda01a0042 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:03:11.237394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-0221629a-5512-43d1-a300-39cda01a0042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:03:11.339503 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221629a-5512-43d1-a300-39cda01a0042 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:03:11.409738 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:03:11.411656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a29e2c-65fd-4590-84cb-a8fd277378b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:03:11.412519 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a29e2c-65fd-4590-84cb-a8fd277378b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:03:11.412989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a29e2c-65fd-4590-84cb-a8fd277378b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:03:11.413405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a29e2c-65fd-4590-84cb-a8fd277378b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:03:11.415811 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-55a29e2c-65fd-4590-84cb-a8fd277378b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:03:11.524658 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a29e2c-65fd-4590-84cb-a8fd277378b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:03:11.526130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a29e2c-65fd-4590-84cb-a8fd277378b2 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:03:11.544854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 579/1164] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:03:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:03:13.775107 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f86a67-2f3d-48a7-9b06-f48531c0f928 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:03:13.775682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f86a67-2f3d-48a7-9b06-f48531c0f928 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:03:13.775682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f86a67-2f3d-48a7-9b06-f48531c0f928 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:03:13.777070 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 586/1165] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:03:13.782216 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997aa722-5925-44db-98d8-4ef769e278b8 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:03:13.782682 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997aa722-5925-44db-98d8-4ef769e278b8 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:03:13.782804 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997aa722-5925-44db-98d8-4ef769e278b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:03:13.849027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-997aa722-5925-44db-98d8-4ef769e278b8 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:03:13.850217 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-997aa722-5925-44db-98d8-4ef769e278b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:03:13.902815 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:03:13.946048 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 580/1166] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:03:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:03:14.239899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f86a67-2f3d-48a7-9b06-f48531c0f928 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:03:14.241484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a3f86a67-2f3d-48a7-9b06-f48531c0f928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:03:14.279105 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:03:14.347434 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f86a67-2f3d-48a7-9b06-f48531c0f928 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:03:14.421674 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1c91f-1841-48a5-a8ef-79daf260932f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:03:14.422209 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1c91f-1841-48a5-a8ef-79daf260932f service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:03:14.422495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1c91f-1841-48a5-a8ef-79daf260932f service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:03:14.422777 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1c91f-1841-48a5-a8ef-79daf260932f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:03:14.424368 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-c8e1c91f-1841-48a5-a8ef-79daf260932f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:03:14.506021 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:03:14.535025 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e1c91f-1841-48a5-a8ef-79daf260932f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:03:14.536981 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e1c91f-1841-48a5-a8ef-79daf260932f service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:03:14.555649 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 587/1167] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:03:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:03:16.902179 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a62e6-cdc4-4ea0-8797-283ad5aa5737 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:03:16.902750 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a62e6-cdc4-4ea0-8797-283ad5aa5737 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:03:16.902750 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a62e6-cdc4-4ea0-8797-283ad5aa5737 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:03:16.904062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 581/1168] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 00:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:03:16.909290 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217442b-4f69-4f1d-90ec-f571b4d330c8 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:03:16.909692 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217442b-4f69-4f1d-90ec-f571b4d330c8 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:03:16.909852 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217442b-4f69-4f1d-90ec-f571b4d330c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:03:16.975766 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-f217442b-4f69-4f1d-90ec-f571b4d330c8 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:03:16.977027 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f217442b-4f69-4f1d-90ec-f571b4d330c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:03:17.073728 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 588/1169] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 00:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:03:17.362740 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a62e6-cdc4-4ea0-8797-283ad5aa5737 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:03:17.364338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-091a62e6-cdc4-4ea0-8797-283ad5aa5737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:03:17.466239 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a62e6-cdc4-4ea0-8797-283ad5aa5737 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:03:17.538008 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc07db1-4b6e-465e-b26d-1a495a85dd10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:03:17.538607 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc07db1-4b6e-465e-b26d-1a495a85dd10 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:03:17.538892 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc07db1-4b6e-465e-b26d-1a495a85dd10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:03:17.539174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc07db1-4b6e-465e-b26d-1a495a85dd10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:03:17.540796 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-0bc07db1-4b6e-465e-b26d-1a495a85dd10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:03:17.549144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:03:17.649002 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc07db1-4b6e-465e-b26d-1a495a85dd10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:03:17.650431 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc07db1-4b6e-465e-b26d-1a495a85dd10 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:03:17.668860 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 582/1170] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:03:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:03:27.072273 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=82 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:03:27.666541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=50 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:04:30.449272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d20c40-7706-43de-af69-df73b699b791 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:04:30.449823 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d20c40-7706-43de-af69-df73b699b791 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:04:30.449823 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d20c40-7706-43de-af69-df73b699b791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:04:30.526581 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-a9d20c40-7706-43de-af69-df73b699b791 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:04:30.527801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a9d20c40-7706-43de-af69-df73b699b791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:04:30.623353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 589/1171] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:04:30.631864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490efb5-63e6-45f5-9ac0-fa1781cd7837 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:04:30.632359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490efb5-63e6-45f5-9ac0-fa1781cd7837 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:04:30.632478 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490efb5-63e6-45f5-9ac0-fa1781cd7837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:04:30.711534 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-b490efb5-63e6-45f5-9ac0-fa1781cd7837 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:04:30.713151 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b490efb5-63e6-45f5-9ac0-fa1781cd7837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:04:30.808740 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 583/1172] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:04:30.813492 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d20c40-7706-43de-af69-df73b699b791 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:04:30.814706 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a9d20c40-7706-43de-af69-df73b699b791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:04:30.826695 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d20c40-7706-43de-af69-df73b699b791 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:04:30.889583 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:04:30.922929 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd794512-d307-4a2b-a5d1-e5772cc04dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:04:30.923530 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd794512-d307-4a2b-a5d1-e5772cc04dcc admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:04:30.923811 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd794512-d307-4a2b-a5d1-e5772cc04dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:04:30.924085 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd794512-d307-4a2b-a5d1-e5772cc04dcc admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:04:30.924759 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd794512-d307-4a2b-a5d1-e5772cc04dcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:04:30.925062 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd794512-d307-4a2b-a5d1-e5772cc04dcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:04:30.926057 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: WARNING py.warnings [None req-bd794512-d307-4a2b-a5d1-e5772cc04dcc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:04:30.926057 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: warnings.warn(msg) Oct 23 00:04:30.926057 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: Oct 23 00:04:30.927385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd794512-d307-4a2b-a5d1-e5772cc04dcc admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:04:30.939102 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 590/1173] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:04:30.949228 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490efb5-63e6-45f5-9ac0-fa1781cd7837 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:04:30.958360 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490efb5-63e6-45f5-9ac0-fa1781cd7837 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:04:31.018723 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:04:31.052634 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88839c3-dfd4-48c3-ae22-4dfc1b62d783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:04:31.053243 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88839c3-dfd4-48c3-ae22-4dfc1b62d783 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:04:31.053523 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88839c3-dfd4-48c3-ae22-4dfc1b62d783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:04:31.053798 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88839c3-dfd4-48c3-ae22-4dfc1b62d783 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:04:31.054417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88839c3-dfd4-48c3-ae22-4dfc1b62d783 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:04:31.055526 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: WARNING py.warnings [None req-c88839c3-dfd4-48c3-ae22-4dfc1b62d783 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:04:31.055526 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: warnings.warn(msg) Oct 23 00:04:31.055526 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: Oct 23 00:04:31.056879 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88839c3-dfd4-48c3-ae22-4dfc1b62d783 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:04:31.115502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 584/1174] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:04:30 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:04:31.119614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd794512-d307-4a2b-a5d1-e5772cc04dcc admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:04:31.121141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-bd794512-d307-4a2b-a5d1-e5772cc04dcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:04:31.154386 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd794512-d307-4a2b-a5d1-e5772cc04dcc admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:04:31.247996 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5780a1-fce7-44ee-8257-1b884763cb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:04:31.248626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5780a1-fce7-44ee-8257-1b884763cb45 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:04:31.248909 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5780a1-fce7-44ee-8257-1b884763cb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:04:31.249179 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5780a1-fce7-44ee-8257-1b884763cb45 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:04:31.249805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5780a1-fce7-44ee-8257-1b884763cb45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:04:31.250793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: WARNING py.warnings [None req-de5780a1-fce7-44ee-8257-1b884763cb45 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:04:31.250793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: warnings.warn(msg) Oct 23 00:04:31.250793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: Oct 23 00:04:31.251988 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5780a1-fce7-44ee-8257-1b884763cb45 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:04:31.294664 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 591/1175] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:04:31 2022] POST /identity/v3/users => generated 290 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:04:31.298694 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88839c3-dfd4-48c3-ae22-4dfc1b62d783 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:04:31.300244 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c88839c3-dfd4-48c3-ae22-4dfc1b62d783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:04:31.364689 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88839c3-dfd4-48c3-ae22-4dfc1b62d783 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:04:31.448720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:04:31.459834 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9c31f-a5e4-4040-809a-a2c5ef25d488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:04:31.460608 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9c31f-a5e4-4040-809a-a2c5ef25d488 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:04:31.460901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9c31f-a5e4-4040-809a-a2c5ef25d488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:04:31.461182 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b9c31f-a5e4-4040-809a-a2c5ef25d488 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:04:31.461831 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b9c31f-a5e4-4040-809a-a2c5ef25d488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:04:31.463456 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b9c31f-a5e4-4040-809a-a2c5ef25d488 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:04:31.477675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 585/1176] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:04:31.481790 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5780a1-fce7-44ee-8257-1b884763cb45 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:04:31.483342 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-de5780a1-fce7-44ee-8257-1b884763cb45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:04:31.495106 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5780a1-fce7-44ee-8257-1b884763cb45 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:04:31.520042 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:04:31.590958 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817a35d-fb78-4912-9e90-4f0d550cd957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:04:31.591563 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817a35d-fb78-4912-9e90-4f0d550cd957 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:04:31.591843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817a35d-fb78-4912-9e90-4f0d550cd957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:04:31.592117 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817a35d-fb78-4912-9e90-4f0d550cd957 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:04:31.592792 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817a35d-fb78-4912-9e90-4f0d550cd957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:04:31.594368 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817a35d-fb78-4912-9e90-4f0d550cd957 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:04:31.606675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 592/1177] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:04:31.610864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9c31f-a5e4-4040-809a-a2c5ef25d488 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:04:31.612422 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-e5b9c31f-a5e4-4040-809a-a2c5ef25d488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:04:31.645126 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b9c31f-a5e4-4040-809a-a2c5ef25d488 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:04:31.736301 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed5a356-19ba-4659-b600-17b41df4f5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:04:31.736728 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5a356-19ba-4659-b600-17b41df4f5c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:04:31.737137 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5a356-19ba-4659-b600-17b41df4f5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:04:31.737271 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5a356-19ba-4659-b600-17b41df4f5c8 admin admin] PATH_INFO: `/v3/projects/a66ffc8a3611446a836ee98fb4236bc3/users/a117121dba81435e89c25d444fbe89e7/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:04:31.760814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed5a356-19ba-4659-b600-17b41df4f5c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a66ffc8a3611446a836ee98fb4236bc3, user_id=a117121dba81435e89c25d444fbe89e7, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:04:31.763489 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed5a356-19ba-4659-b600-17b41df4f5c8 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:04:31.787689 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 586/1178] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:04:31 2022] PUT /identity/v3/projects/a66ffc8a3611446a836ee98fb4236bc3/users/a117121dba81435e89c25d444fbe89e7/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:04:31.798891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382539a3-2cc0-45cd-ae56-48ca976ac429 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:04:31.799187 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382539a3-2cc0-45cd-ae56-48ca976ac429 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:04:31.799469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382539a3-2cc0-45cd-ae56-48ca976ac429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:04:31.857097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-382539a3-2cc0-45cd-ae56-48ca976ac429 None None] MFA Rules not processed for user `a117121dba81435e89c25d444fbe89e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:04:31.858242 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-382539a3-2cc0-45cd-ae56-48ca976ac429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:04:31.935963 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 593/1179] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:04:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:04:31.949191 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c1e53-5fde-4179-82c1-1d4434ce5795 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:04:31.949483 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c1e53-5fde-4179-82c1-1d4434ce5795 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:04:31.949758 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c1e53-5fde-4179-82c1-1d4434ce5795 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:04:31.951117 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 587/1180] 10.222.0.13 () {54 vars in 963 bytes} [Sun Oct 23 00:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:04:31.955740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021f703-2ce3-4231-bb69-63360a5eb240 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:04:31.956034 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021f703-2ce3-4231-bb69-63360a5eb240 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:04:31.956316 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021f703-2ce3-4231-bb69-63360a5eb240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:04:31.988733 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:04:32.035277 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-e021f703-2ce3-4231-bb69-63360a5eb240 None None] MFA Rules not processed for user `bd170de478a344fc98951697c7a6298d`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:04:32.037139 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e021f703-2ce3-4231-bb69-63360a5eb240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:04:32.113461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 594/1181] 10.222.0.13 () {58 vars in 1060 bytes} [Sun Oct 23 00:04:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:04:32.121536 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c1e53-5fde-4179-82c1-1d4434ce5795 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:04:32.123130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-3d4c1e53-5fde-4179-82c1-1d4434ce5795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:04:32.172773 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:04:32.226222 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c1e53-5fde-4179-82c1-1d4434ce5795 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:04:32.299427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068f0ac-d0a3-4355-9403-050100ab02bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:04:32.300026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068f0ac-d0a3-4355-9403-050100ab02bd service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:04:32.300309 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068f0ac-d0a3-4355-9403-050100ab02bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:04:32.300618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8068f0ac-d0a3-4355-9403-050100ab02bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:04:32.302679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8068f0ac-d0a3-4355-9403-050100ab02bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:04:32.400689 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8068f0ac-d0a3-4355-9403-050100ab02bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:04:32.402032 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8068f0ac-d0a3-4355-9403-050100ab02bd service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:04:32.405512 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:04:32.420012 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 588/1182] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:04:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:04:32.466267 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e12dd-4731-4020-b123-01dbf7395d52 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:04:32.466559 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e12dd-4731-4020-b123-01dbf7395d52 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:04:32.466837 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e12dd-4731-4020-b123-01dbf7395d52 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:04:32.468088 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 595/1183] 10.222.0.13 () {52 vars in 830 bytes} [Sun Oct 23 00:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:04:32.471795 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068f0ac-d0a3-4355-9403-050100ab02bd service cinder] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:04:32.473062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8068f0ac-d0a3-4355-9403-050100ab02bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:04:32.570198 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8068f0ac-d0a3-4355-9403-050100ab02bd service cinder] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:04:32.661437 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc863a06-05ac-4675-b99d-9a936a49c709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:04:32.662088 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc863a06-05ac-4675-b99d-9a936a49c709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:04:32.662240 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc863a06-05ac-4675-b99d-9a936a49c709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:04:32.662510 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc863a06-05ac-4675-b99d-9a936a49c709 admin admin] PATH_INFO: `/v3/users/a117121dba81435e89c25d444fbe89e7` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:04:32.675036 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc863a06-05ac-4675-b99d-9a936a49c709 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a117121dba81435e89c25d444fbe89e7)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:04:32.676059 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: WARNING py.warnings [None req-fc863a06-05ac-4675-b99d-9a936a49c709 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:04:32.676059 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: warnings.warn(msg) Oct 23 00:04:32.676059 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: Oct 23 00:04:32.677879 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc863a06-05ac-4675-b99d-9a936a49c709 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:04:32.687396 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:04:32.742655 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fc863a06-05ac-4675-b99d-9a936a49c709 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a117121dba81435e89c25d444fbe89e7'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:04:32.743197 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fc863a06-05ac-4675-b99d-9a936a49c709 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a117121dba81435e89c25d444fbe89e7'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:04:32.757333 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fc863a06-05ac-4675-b99d-9a936a49c709 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a117121dba81435e89c25d444fbe89e7'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:04:32.779880 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fc863a06-05ac-4675-b99d-9a936a49c709 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a117121dba81435e89c25d444fbe89e7'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:04:32.788830 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fc863a06-05ac-4675-b99d-9a936a49c709 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a117121dba81435e89c25d444fbe89e7'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:04:32.794595 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 589/1184] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:04:32 2022] DELETE /identity/v3/users/a117121dba81435e89c25d444fbe89e7 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:04:32.799132 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33624ec6-c7d5-4320-aea1-72596c959a0e None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:04:32.799624 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33624ec6-c7d5-4320-aea1-72596c959a0e None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:04:32.799737 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33624ec6-c7d5-4320-aea1-72596c959a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:04:32.857435 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-33624ec6-c7d5-4320-aea1-72596c959a0e None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:04:32.858660 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-33624ec6-c7d5-4320-aea1-72596c959a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:04:32.933106 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:04:32.946404 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 596/1185] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:04:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:04:32.956156 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc863a06-05ac-4675-b99d-9a936a49c709 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:04:32.957430 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-fc863a06-05ac-4675-b99d-9a936a49c709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:04:33.055915 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc863a06-05ac-4675-b99d-9a936a49c709 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:04:33.113759 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:04:33.134808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51ea10-98a4-4179-a0b0-d86b60b84a7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:04:33.135421 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51ea10-98a4-4179-a0b0-d86b60b84a7f service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:04:33.135705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51ea10-98a4-4179-a0b0-d86b60b84a7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:04:33.135984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51ea10-98a4-4179-a0b0-d86b60b84a7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:04:33.137638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-1b51ea10-98a4-4179-a0b0-d86b60b84a7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:04:33.245168 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51ea10-98a4-4179-a0b0-d86b60b84a7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:04:33.246521 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51ea10-98a4-4179-a0b0-d86b60b84a7f service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:04:33.264400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 590/1186] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:04:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:04:33.627603 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33624ec6-c7d5-4320-aea1-72596c959a0e None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:04:33.628928 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-33624ec6-c7d5-4320-aea1-72596c959a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:04:33.640341 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33624ec6-c7d5-4320-aea1-72596c959a0e None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:04:33.743155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75eaf5c-13e6-4476-bb65-e136f2fe993e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:04:33.743796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75eaf5c-13e6-4476-bb65-e136f2fe993e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:04:33.744084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75eaf5c-13e6-4476-bb65-e136f2fe993e admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:04:33.744361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75eaf5c-13e6-4476-bb65-e136f2fe993e admin admin] PATH_INFO: `/v3/projects/a66ffc8a3611446a836ee98fb4236bc3` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:04:33.754677 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75eaf5c-13e6-4476-bb65-e136f2fe993e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a66ffc8a3611446a836ee98fb4236bc3)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:04:33.756247 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: WARNING py.warnings [None req-f75eaf5c-13e6-4476-bb65-e136f2fe993e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:04:33.756247 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: warnings.warn(msg) Oct 23 00:04:33.756247 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: Oct 23 00:04:33.757485 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75eaf5c-13e6-4476-bb65-e136f2fe993e admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:04:33.815915 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f75eaf5c-13e6-4476-bb65-e136f2fe993e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a66ffc8a3611446a836ee98fb4236bc3', 'request_id': 'req-f75eaf5c-13e6-4476-bb65-e136f2fe993e'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:04:33.829170 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f75eaf5c-13e6-4476-bb65-e136f2fe993e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a66ffc8a3611446a836ee98fb4236bc3', 'request_id': 'req-f75eaf5c-13e6-4476-bb65-e136f2fe993e'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:04:33.838551 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f75eaf5c-13e6-4476-bb65-e136f2fe993e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a66ffc8a3611446a836ee98fb4236bc3', 'request_id': 'req-f75eaf5c-13e6-4476-bb65-e136f2fe993e'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:04:33.861090 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f75eaf5c-13e6-4476-bb65-e136f2fe993e admin admin] The token cache is being invalidate because project a66ffc8a3611446a836ee98fb4236bc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:04:33.861376 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f75eaf5c-13e6-4476-bb65-e136f2fe993e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:04:33.861801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f75eaf5c-13e6-4476-bb65-e136f2fe993e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:04:33.863190 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 597/1187] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:04:33 2022] DELETE /identity/v3/projects/a66ffc8a3611446a836ee98fb4236bc3 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:04:43.262201 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=62 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:04:43.859077 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:05:56.085548 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990c740-350e-4fea-a748-5b57edea1520 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:05:56.086148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990c740-350e-4fea-a748-5b57edea1520 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:05:56.086148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990c740-350e-4fea-a748-5b57edea1520 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:05:56.087745 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 591/1188] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 00:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:05:56.093451 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb677fd4-9558-4c8d-8eaf-3db5699e1270 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:05:56.093874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb677fd4-9558-4c8d-8eaf-3db5699e1270 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:05:56.094062 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb677fd4-9558-4c8d-8eaf-3db5699e1270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:05:56.178171 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-eb677fd4-9558-4c8d-8eaf-3db5699e1270 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:05:56.179394 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-eb677fd4-9558-4c8d-8eaf-3db5699e1270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:05:56.285291 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 598/1189] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 00:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:05:56.442904 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b990c740-350e-4fea-a748-5b57edea1520 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:05:56.445054 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b990c740-350e-4fea-a748-5b57edea1520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:05:56.564384 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b990c740-350e-4fea-a748-5b57edea1520 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:05:56.569898 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae643c-a740-4038-afd9-363916276347 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:05:56.570507 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae643c-a740-4038-afd9-363916276347 service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:05:56.570789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae643c-a740-4038-afd9-363916276347 service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:05:56.571092 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae643c-a740-4038-afd9-363916276347 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:05:56.579731 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae643c-a740-4038-afd9-363916276347 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:05:56.581354 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae643c-a740-4038-afd9-363916276347 service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:05:56.591374 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 592/1190] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:05:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:05:57.680439 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f277f-759a-48c5-a3a0-87d5b8a8b7a9 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:05:57.680968 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f277f-759a-48c5-a3a0-87d5b8a8b7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:05:57.680968 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f277f-759a-48c5-a3a0-87d5b8a8b7a9 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:05:57.682341 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 599/1191] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 00:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:05:57.687885 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288baac-1f9a-42c9-ba4c-8ee00a369a55 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:05:57.688312 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288baac-1f9a-42c9-ba4c-8ee00a369a55 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:05:57.688441 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288baac-1f9a-42c9-ba4c-8ee00a369a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:05:57.707740 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-8288baac-1f9a-42c9-ba4c-8ee00a369a55 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:05:57.709002 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8288baac-1f9a-42c9-ba4c-8ee00a369a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:05:57.717415 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 593/1192] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 00:05:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:05:57.869156 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f277f-759a-48c5-a3a0-87d5b8a8b7a9 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:05:57.884108 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f277f-759a-48c5-a3a0-87d5b8a8b7a9 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:05:57.889615 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59048146-ff01-4483-ac10-08bd0de060bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:05:57.890218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59048146-ff01-4483-ac10-08bd0de060bd service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:05:57.890502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59048146-ff01-4483-ac10-08bd0de060bd service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:05:57.890779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59048146-ff01-4483-ac10-08bd0de060bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:05:57.899105 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59048146-ff01-4483-ac10-08bd0de060bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:05:57.900715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59048146-ff01-4483-ac10-08bd0de060bd service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:05:57.909995 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 600/1193] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:05:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:05:58.618822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f05ee-620d-4fca-83e7-fac3782a3f16 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:05:58.619114 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f05ee-620d-4fca-83e7-fac3782a3f16 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:05:58.619399 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f05ee-620d-4fca-83e7-fac3782a3f16 None None] PATH_INFO: `/v3` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:05:58.620829 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 594/1194] 10.222.0.15 () {54 vars in 925 bytes} [Sun Oct 23 00:05:58 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:05:58.627234 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d5324-d2a3-4ee8-8ad8-00bddb7437e2 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:05:58.627524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d5324-d2a3-4ee8-8ad8-00bddb7437e2 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:05:58.627803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d5324-d2a3-4ee8-8ad8-00bddb7437e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:05:58.687868 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-0e9d5324-d2a3-4ee8-8ad8-00bddb7437e2 None None] MFA Rules not processed for user `24108b6355414f64ba5e7bf84011eee6`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:05:58.689111 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-0e9d5324-d2a3-4ee8-8ad8-00bddb7437e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:05:58.706208 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:05:58.763882 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 601/1195] 10.222.0.15 () {58 vars in 1015 bytes} [Sun Oct 23 00:05:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:06:02.408310 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e03630-8a56-4e20-9d0d-5172e6894e4f None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:06:02.408755 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e03630-8a56-4e20-9d0d-5172e6894e4f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:06:02.408914 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e03630-8a56-4e20-9d0d-5172e6894e4f None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:06:02.410243 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 595/1196] 10.222.0.13 () {54 vars in 961 bytes} [Sun Oct 23 00:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:06:02.415131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4449848-3964-4e2d-945d-9987bac09689 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:06:02.415574 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4449848-3964-4e2d-945d-9987bac09689 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:06:02.415695 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4449848-3964-4e2d-945d-9987bac09689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:06:02.436260 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-a4449848-3964-4e2d-945d-9987bac09689 None None] MFA Rules not processed for user `24108b6355414f64ba5e7bf84011eee6`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:06:02.437616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a4449848-3964-4e2d-945d-9987bac09689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:06:02.445888 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 602/1197] 10.222.0.13 () {58 vars in 1058 bytes} [Sun Oct 23 00:06:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:06:02.453560 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e03630-8a56-4e20-9d0d-5172e6894e4f None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:06:02.467988 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e03630-8a56-4e20-9d0d-5172e6894e4f None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:06:02.472927 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a287e1-3e33-43db-9782-70df4de70969 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24108b6355414f64ba5e7bf84011eee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:06:02.473517 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a287e1-3e33-43db-9782-70df4de70969 service placement] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:06:02.473805 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a287e1-3e33-43db-9782-70df4de70969 service placement] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:06:02.474081 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a287e1-3e33-43db-9782-70df4de70969 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:06:02.482130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a287e1-3e33-43db-9782-70df4de70969 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:06:02.484381 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a287e1-3e33-43db-9782-70df4de70969 service placement] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:06:02.493758 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 596/1198] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:06:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:06:09.362911 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfedd9c-bdf7-4699-9ea0-21a29dbc5b65 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:06:09.363491 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfedd9c-bdf7-4699-9ea0-21a29dbc5b65 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:06:09.363491 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfedd9c-bdf7-4699-9ea0-21a29dbc5b65 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:06:09.364740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 603/1199] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 00:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:06:09.369512 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a67328-ac5c-4f29-be80-2eb832def8a3 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:06:09.369961 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a67328-ac5c-4f29-be80-2eb832def8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:06:09.370100 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a67328-ac5c-4f29-be80-2eb832def8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:06:09.389737 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-26a67328-ac5c-4f29-be80-2eb832def8a3 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:06:09.391020 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-26a67328-ac5c-4f29-be80-2eb832def8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:06:09.398774 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 597/1200] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 00:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:06:09.554931 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfedd9c-bdf7-4699-9ea0-21a29dbc5b65 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:06:09.569491 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfedd9c-bdf7-4699-9ea0-21a29dbc5b65 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:06:09.574994 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b163b9-1841-4383-8d4e-71dda1bce4cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:06:09.575589 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b163b9-1841-4383-8d4e-71dda1bce4cc service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:06:09.575873 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b163b9-1841-4383-8d4e-71dda1bce4cc service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:06:09.576161 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b163b9-1841-4383-8d4e-71dda1bce4cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:06:09.584051 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b163b9-1841-4383-8d4e-71dda1bce4cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:06:09.585646 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b163b9-1841-4383-8d4e-71dda1bce4cc service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:06:09.595936 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 604/1201] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:06:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:06:10.680592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a218556-3f5d-4413-8d2a-18b2ef69a461 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:06:10.681196 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a218556-3f5d-4413-8d2a-18b2ef69a461 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:06:10.681196 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a218556-3f5d-4413-8d2a-18b2ef69a461 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:06:10.682490 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 598/1202] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 00:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:06:10.688040 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa5bba-f497-4f6a-9f44-17d5a9ba4ecf None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:06:10.688614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa5bba-f497-4f6a-9f44-17d5a9ba4ecf None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:06:10.688614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa5bba-f497-4f6a-9f44-17d5a9ba4ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:06:10.708855 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-33fa5bba-f497-4f6a-9f44-17d5a9ba4ecf None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:06:10.710062 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-33fa5bba-f497-4f6a-9f44-17d5a9ba4ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:06:10.719339 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 605/1203] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 00:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:06:10.869936 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a218556-3f5d-4413-8d2a-18b2ef69a461 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:06:10.885016 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a218556-3f5d-4413-8d2a-18b2ef69a461 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:06:10.891521 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfee2f4-dcda-447f-94e8-dbc8e1da54c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:06:10.892109 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfee2f4-dcda-447f-94e8-dbc8e1da54c8 service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:06:10.892390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfee2f4-dcda-447f-94e8-dbc8e1da54c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:06:10.892701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfee2f4-dcda-447f-94e8-dbc8e1da54c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:06:10.900559 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfee2f4-dcda-447f-94e8-dbc8e1da54c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:06:10.902118 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfee2f4-dcda-447f-94e8-dbc8e1da54c8 service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:06:10.911909 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 599/1204] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:06:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:06:14.789650 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f0b04-57f9-4aa9-a312-90d4a43ef639 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:06:14.790222 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f0b04-57f9-4aa9-a312-90d4a43ef639 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:06:14.790222 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f0b04-57f9-4aa9-a312-90d4a43ef639 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:06:14.791664 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 606/1205] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:06:14.797698 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d6286-c727-48bf-aa42-42c01791d7d0 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:06:14.798083 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d6286-c727-48bf-aa42-42c01791d7d0 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:06:14.798253 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d6286-c727-48bf-aa42-42c01791d7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:06:14.817317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-4c6d6286-c727-48bf-aa42-42c01791d7d0 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:06:14.818510 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-4c6d6286-c727-48bf-aa42-42c01791d7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:06:14.826309 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 600/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:06:15.116127 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f0b04-57f9-4aa9-a312-90d4a43ef639 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:06:15.117852 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e66f0b04-57f9-4aa9-a312-90d4a43ef639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:06:15.195292 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f0b04-57f9-4aa9-a312-90d4a43ef639 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:06:15.198911 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc10359-9a2d-45da-bfac-fe3ab95b87c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:06:15.199513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc10359-9a2d-45da-bfac-fe3ab95b87c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:06:15.199798 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc10359-9a2d-45da-bfac-fe3ab95b87c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:06:15.200078 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc10359-9a2d-45da-bfac-fe3ab95b87c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:06:15.209375 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc10359-9a2d-45da-bfac-fe3ab95b87c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:06:15.210825 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc10359-9a2d-45da-bfac-fe3ab95b87c4 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:06:15.221157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 607/1207] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:06:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:06:17.387409 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1a628-cb8f-4688-885e-99651a72249d None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:06:17.387920 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1a628-cb8f-4688-885e-99651a72249d None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:06:17.387920 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1a628-cb8f-4688-885e-99651a72249d None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:06:17.389357 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 601/1208] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:06:17.394491 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546af5aa-eda8-4d5d-b0a5-41b7c55a65ae None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:06:17.394905 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546af5aa-eda8-4d5d-b0a5-41b7c55a65ae None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:06:17.395046 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546af5aa-eda8-4d5d-b0a5-41b7c55a65ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:06:17.417261 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-546af5aa-eda8-4d5d-b0a5-41b7c55a65ae None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:06:17.418462 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-546af5aa-eda8-4d5d-b0a5-41b7c55a65ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:06:17.427607 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 608/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:06:17.717116 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1a628-cb8f-4688-885e-99651a72249d None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:06:17.727972 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1a628-cb8f-4688-885e-99651a72249d None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:06:17.731494 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f42e41-f401-4e91-bb11-5cddf605661b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:06:17.732093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f42e41-f401-4e91-bb11-5cddf605661b service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:06:17.732414 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f42e41-f401-4e91-bb11-5cddf605661b service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:06:17.732811 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f42e41-f401-4e91-bb11-5cddf605661b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:06:17.741586 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f42e41-f401-4e91-bb11-5cddf605661b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:06:17.742991 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f42e41-f401-4e91-bb11-5cddf605661b service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:06:17.752988 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 602/1210] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:06:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:06:21.266619 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1fd32-2008-4943-8114-e0bb3d1ae3b4 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:06:21.267181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1fd32-2008-4943-8114-e0bb3d1ae3b4 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:06:21.267181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1fd32-2008-4943-8114-e0bb3d1ae3b4 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:06:21.268552 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 609/1211] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:06:21.273880 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26056fe-1bc6-4d91-89db-d4dae79d4597 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:06:21.274309 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26056fe-1bc6-4d91-89db-d4dae79d4597 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:06:21.274445 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26056fe-1bc6-4d91-89db-d4dae79d4597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:06:21.294031 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-a26056fe-1bc6-4d91-89db-d4dae79d4597 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:06:21.295835 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a26056fe-1bc6-4d91-89db-d4dae79d4597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:06:21.303721 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 603/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:06:21.604682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1fd32-2008-4943-8114-e0bb3d1ae3b4 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:06:21.606494 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-2ef1fd32-2008-4943-8114-e0bb3d1ae3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:06:21.671822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:06:21.685993 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1fd32-2008-4943-8114-e0bb3d1ae3b4 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:06:21.689193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85542d4-ea9d-4d39-82d5-08b58417ae7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:06:21.689806 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85542d4-ea9d-4d39-82d5-08b58417ae7b service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:06:21.690094 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85542d4-ea9d-4d39-82d5-08b58417ae7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:06:21.690373 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85542d4-ea9d-4d39-82d5-08b58417ae7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:06:21.699843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85542d4-ea9d-4d39-82d5-08b58417ae7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:06:21.702056 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85542d4-ea9d-4d39-82d5-08b58417ae7b service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:06:21.711539 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 610/1213] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:06:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:06:23.850014 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440ece9-5f31-477e-87cc-16f801420edd None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:06:23.850561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440ece9-5f31-477e-87cc-16f801420edd None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:06:23.850561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440ece9-5f31-477e-87cc-16f801420edd None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:06:23.851959 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 604/1214] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:06:23.857114 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9df6ec-b50f-4780-bc20-c292e1839ed8 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:06:23.858278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9df6ec-b50f-4780-bc20-c292e1839ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:06:23.858278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9df6ec-b50f-4780-bc20-c292e1839ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:06:23.877306 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-ea9df6ec-b50f-4780-bc20-c292e1839ed8 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:06:23.878500 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ea9df6ec-b50f-4780-bc20-c292e1839ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:06:23.886149 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 611/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:06:24.186304 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440ece9-5f31-477e-87cc-16f801420edd None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:06:24.188118 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-5440ece9-5f31-477e-87cc-16f801420edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:06:24.195631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=84 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:06:24.201864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440ece9-5f31-477e-87cc-16f801420edd None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:06:24.205417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd3437-649b-43bf-bae9-045de05002b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:06:24.206013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd3437-649b-43bf-bae9-045de05002b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:06:24.206299 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd3437-649b-43bf-bae9-045de05002b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:06:24.206576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd3437-649b-43bf-bae9-045de05002b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:06:24.215774 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd3437-649b-43bf-bae9-045de05002b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:06:24.217993 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd3437-649b-43bf-bae9-045de05002b5 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:06:24.227248 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 605/1216] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:06:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:06:33.872506 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=11 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:06:34.223139 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=5 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:46.660736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3989f66-836e-4044-bb08-e317662e6ed5 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:46.661437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3989f66-836e-4044-bb08-e317662e6ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:46.661437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3989f66-836e-4044-bb08-e317662e6ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:46.687915 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-b3989f66-836e-4044-bb08-e317662e6ed5 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:46.689164 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b3989f66-836e-4044-bb08-e317662e6ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:46.704161 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 612/1217] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:46.705258 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb8fd3-1a13-48c8-93c2-c13cdc93cdc2 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:46.705772 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb8fd3-1a13-48c8-93c2-c13cdc93cdc2 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:46.705878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb8fd3-1a13-48c8-93c2-c13cdc93cdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:46.718352 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c78ef-6195-40e1-a639-8d9d726a07bb None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:46.718760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c78ef-6195-40e1-a639-8d9d726a07bb None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:46.718956 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c78ef-6195-40e1-a639-8d9d726a07bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:46.744139 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-0cfb8fd3-1a13-48c8-93c2-c13cdc93cdc2 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:46.745563 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-0cfb8fd3-1a13-48c8-93c2-c13cdc93cdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:46.752704 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1b9c78ef-6195-40e1-a639-8d9d726a07bb None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:46.753945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1b9c78ef-6195-40e1-a639-8d9d726a07bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:46.755446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 606/1218] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:07:46.760265 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd9159-86b1-4dfc-a069-39a5042a3967 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:46.760661 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd9159-86b1-4dfc-a069-39a5042a3967 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:46.760954 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd9159-86b1-4dfc-a069-39a5042a3967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:46.770877 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 613/1219] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:46.772377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477a43e-7279-4533-9bdf-d25930d8f998 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:46.772698 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477a43e-7279-4533-9bdf-d25930d8f998 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:46.772973 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477a43e-7279-4533-9bdf-d25930d8f998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:46.800239 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-8edd9159-86b1-4dfc-a069-39a5042a3967 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:46.801791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8edd9159-86b1-4dfc-a069-39a5042a3967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:46.814754 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-2477a43e-7279-4533-9bdf-d25930d8f998 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:46.815976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-2477a43e-7279-4533-9bdf-d25930d8f998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:46.825666 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 607/1220] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:46.827225 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa33328-8da3-4a65-81c1-612f76295fcc None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:46.827566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa33328-8da3-4a65-81c1-612f76295fcc None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:46.827855 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa33328-8da3-4a65-81c1-612f76295fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:46.843998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 614/1221] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:46.845597 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6c2ba-926d-4e21-a1f2-71e146cd765a None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:46.845884 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6c2ba-926d-4e21-a1f2-71e146cd765a None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:46.846161 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6c2ba-926d-4e21-a1f2-71e146cd765a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:46.863857 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-2aa33328-8da3-4a65-81c1-612f76295fcc None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:46.865106 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2aa33328-8da3-4a65-81c1-612f76295fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:46.882308 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 608/1222] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:46.883467 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa33328-8da3-4a65-81c1-612f76295fcc None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:46.885348 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-89d6c2ba-926d-4e21-a1f2-71e146cd765a None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:46.886819 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-89d6c2ba-926d-4e21-a1f2-71e146cd765a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:46.896252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa33328-8da3-4a65-81c1-612f76295fcc None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:46.898826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 615/1223] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:46.899732 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6c2ba-926d-4e21-a1f2-71e146cd765a None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:46.900853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd71bfc-015f-4c2f-bba9-667e213fc974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:46.901473 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd71bfc-015f-4c2f-bba9-667e213fc974 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:46.901753 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd71bfc-015f-4c2f-bba9-667e213fc974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:46.902021 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd71bfc-015f-4c2f-bba9-667e213fc974 admin admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:46.902667 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd71bfc-015f-4c2f-bba9-667e213fc974 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:07:46.903091 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd71bfc-015f-4c2f-bba9-667e213fc974 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:46.904116 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: WARNING py.warnings [None req-4fd71bfc-015f-4c2f-bba9-667e213fc974 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:07:46.904116 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: warnings.warn(msg) Oct 23 00:07:46.904116 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: Oct 23 00:07:46.905888 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd71bfc-015f-4c2f-bba9-667e213fc974 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:46.911064 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6c2ba-926d-4e21-a1f2-71e146cd765a None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:46.916695 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0e02d-419f-4a28-86ec-6ad70a029e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:46.917311 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0e02d-419f-4a28-86ec-6ad70a029e8a admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:46.917598 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0e02d-419f-4a28-86ec-6ad70a029e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:46.917886 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0e02d-419f-4a28-86ec-6ad70a029e8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:46.918673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0e02d-419f-4a28-86ec-6ad70a029e8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:07:46.918985 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0e02d-419f-4a28-86ec-6ad70a029e8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:46.919106 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 609/1224] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:07:46.920334 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42c9be-81d6-40b4-8430-6c8562f7c8f0 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:46.920588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0e02d-419f-4a28-86ec-6ad70a029e8a admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:46.920740 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42c9be-81d6-40b4-8430-6c8562f7c8f0 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:46.921626 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42c9be-81d6-40b4-8430-6c8562f7c8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:46.941131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 616/1225] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:07:46.946038 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901b989-74c8-48b0-8fdd-7b14230ef1ce None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:46.946331 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901b989-74c8-48b0-8fdd-7b14230ef1ce None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:46.947117 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901b989-74c8-48b0-8fdd-7b14230ef1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:46.948231 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-aa42c9be-81d6-40b4-8430-6c8562f7c8f0 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:46.949788 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-aa42c9be-81d6-40b4-8430-6c8562f7c8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:46.959322 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 610/1226] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:46.960861 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba2bdd-0cb1-4b99-a8f7-5dd9011d859a None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:46.961386 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba2bdd-0cb1-4b99-a8f7-5dd9011d859a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:46.961881 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba2bdd-0cb1-4b99-a8f7-5dd9011d859a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:46.969107 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-6901b989-74c8-48b0-8fdd-7b14230ef1ce None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:46.970616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-6901b989-74c8-48b0-8fdd-7b14230ef1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:46.979167 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 617/1227] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:46.980103 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901b989-74c8-48b0-8fdd-7b14230ef1ce None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:46.985638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-2cba2bdd-0cb1-4b99-a8f7-5dd9011d859a None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:46.987264 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2cba2bdd-0cb1-4b99-a8f7-5dd9011d859a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:46.991221 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901b989-74c8-48b0-8fdd-7b14230ef1ce None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:46.995872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 611/1228] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:46.996018 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cead00d-3544-4f04-afde-e1c56b98fd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:46.997049 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cead00d-3544-4f04-afde-e1c56b98fd62 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:46.997211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba2bdd-0cb1-4b99-a8f7-5dd9011d859a None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:46.997311 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cead00d-3544-4f04-afde-e1c56b98fd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:46.997311 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cead00d-3544-4f04-afde-e1c56b98fd62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:46.997920 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cead00d-3544-4f04-afde-e1c56b98fd62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:07:46.998235 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cead00d-3544-4f04-afde-e1c56b98fd62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.000011 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cead00d-3544-4f04-afde-e1c56b98fd62 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.007705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba2bdd-0cb1-4b99-a8f7-5dd9011d859a None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.012145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48c32a-c574-4f5a-84c4-13cc7a6c4183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.012837 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 618/1229] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:07:47.012972 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48c32a-c574-4f5a-84c4-13cc7a6c4183 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.013069 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48c32a-c574-4f5a-84c4-13cc7a6c4183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.016275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48c32a-c574-4f5a-84c4-13cc7a6c4183 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.016275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae48c32a-c574-4f5a-84c4-13cc7a6c4183 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.016534 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96338e-fcf8-4ba3-afb8-5ae99abb1620 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.016534 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96338e-fcf8-4ba3-afb8-5ae99abb1620 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.016534 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96338e-fcf8-4ba3-afb8-5ae99abb1620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.016772 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae48c32a-c574-4f5a-84c4-13cc7a6c4183 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.035612 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-7e96338e-fcf8-4ba3-afb8-5ae99abb1620 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:47.036856 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-7e96338e-fcf8-4ba3-afb8-5ae99abb1620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:47.047002 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 619/1230] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:47.047851 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96338e-fcf8-4ba3-afb8-5ae99abb1620 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.057944 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 612/1231] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:07:46 2022] POST /identity/v3/projects => generated 403 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:47.058226 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96338e-fcf8-4ba3-afb8-5ae99abb1620 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.059353 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0d51d-42ed-4b2a-98cd-0df148c6220b None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.059645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0d51d-42ed-4b2a-98cd-0df148c6220b None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.060068 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0d51d-42ed-4b2a-98cd-0df148c6220b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.080869 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-79f0d51d-42ed-4b2a-98cd-0df148c6220b None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:47.082081 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-79f0d51d-42ed-4b2a-98cd-0df148c6220b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:47.120256 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3684b8a-a6c6-4da7-9be8-368aeb3be478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.120911 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3684b8a-a6c6-4da7-9be8-368aeb3be478 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.121200 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3684b8a-a6c6-4da7-9be8-368aeb3be478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.121510 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3684b8a-a6c6-4da7-9be8-368aeb3be478 admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.122150 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3684b8a-a6c6-4da7-9be8-368aeb3be478 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.123188 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: WARNING py.warnings [None req-f3684b8a-a6c6-4da7-9be8-368aeb3be478 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:07:47.123188 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: warnings.warn(msg) Oct 23 00:07:47.123188 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: Oct 23 00:07:47.124634 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3684b8a-a6c6-4da7-9be8-368aeb3be478 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.141708 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 613/1232] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:47.143514 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8d794-569e-4a6e-be12-f15500721074 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.144004 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8d794-569e-4a6e-be12-f15500721074 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.144237 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f8d794-569e-4a6e-be12-f15500721074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.153373 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 620/1233] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:07:47 2022] POST /identity/v3/projects => generated 389 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:47.154224 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3684b8a-a6c6-4da7-9be8-368aeb3be478 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.164438 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-88f8d794-569e-4a6e-be12-f15500721074 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:47.165625 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-88f8d794-569e-4a6e-be12-f15500721074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:47.220621 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3684b8a-a6c6-4da7-9be8-368aeb3be478 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.225332 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672e776-e7ee-4277-9978-0b82955643ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.226016 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 614/1234] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:47.226150 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672e776-e7ee-4277-9978-0b82955643ff admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.226234 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672e776-e7ee-4277-9978-0b82955643ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.226461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672e776-e7ee-4277-9978-0b82955643ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.226714 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8d794-569e-4a6e-be12-f15500721074 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.227102 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4672e776-e7ee-4277-9978-0b82955643ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:07:47.227403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4672e776-e7ee-4277-9978-0b82955643ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.228999 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4672e776-e7ee-4277-9978-0b82955643ff admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.236707 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f8d794-569e-4a6e-be12-f15500721074 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.248218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 621/1235] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:07:47.248425 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d926054-309d-4264-9db0-5a1416df09fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.248425 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d926054-309d-4264-9db0-5a1416df09fe admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.248425 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d926054-309d-4264-9db0-5a1416df09fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.249111 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944b354-fadf-44c3-9a97-dba518fc3241 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.249206 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d926054-309d-4264-9db0-5a1416df09fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.249299 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944b354-fadf-44c3-9a97-dba518fc3241 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.249299 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944b354-fadf-44c3-9a97-dba518fc3241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.249456 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d926054-309d-4264-9db0-5a1416df09fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.249456 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d926054-309d-4264-9db0-5a1416df09fe admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.262183 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-7944b354-fadf-44c3-9a97-dba518fc3241 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:47.263427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-7944b354-fadf-44c3-9a97-dba518fc3241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:47.272465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 622/1236] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:47.281191 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e20b74-60be-4bdd-b9fc-3cd1d5059c40 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.281329 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 615/1237] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:07:47 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:47.281440 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e20b74-60be-4bdd-b9fc-3cd1d5059c40 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.281440 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e20b74-60be-4bdd-b9fc-3cd1d5059c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.281719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d926054-309d-4264-9db0-5a1416df09fe admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.284656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:47.293377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:47.299775 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-91e20b74-60be-4bdd-b9fc-3cd1d5059c40 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:47.301176 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-91e20b74-60be-4bdd-b9fc-3cd1d5059c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:47.346283 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d926054-309d-4264-9db0-5a1416df09fe admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.351199 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5354fee-b1e0-4d04-b6d0-b4ff9def01a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.351807 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5354fee-b1e0-4d04-b6d0-b4ff9def01a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.352094 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5354fee-b1e0-4d04-b6d0-b4ff9def01a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.352375 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5354fee-b1e0-4d04-b6d0-b4ff9def01a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.353058 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5354fee-b1e0-4d04-b6d0-b4ff9def01a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.354093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: WARNING py.warnings [None req-c5354fee-b1e0-4d04-b6d0-b4ff9def01a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:07:47.354093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: warnings.warn(msg) Oct 23 00:07:47.354093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: Oct 23 00:07:47.355330 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5354fee-b1e0-4d04-b6d0-b4ff9def01a1 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.368440 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 623/1238] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:47.373302 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e20b74-60be-4bdd-b9fc-3cd1d5059c40 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.376877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 616/1239] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:07:47 2022] POST /identity/v3/users => generated 302 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:47.377675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5354fee-b1e0-4d04-b6d0-b4ff9def01a1 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.386911 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e20b74-60be-4bdd-b9fc-3cd1d5059c40 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.387356 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5354fee-b1e0-4d04-b6d0-b4ff9def01a1 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.392790 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1166a0c-dbe0-47b9-916b-788c3c9034e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.392937 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435342a9-26ef-46b8-a1fa-03b1aeac3928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.393384 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1166a0c-dbe0-47b9-916b-788c3c9034e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.393501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435342a9-26ef-46b8-a1fa-03b1aeac3928 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.393670 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1166a0c-dbe0-47b9-916b-788c3c9034e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.393773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435342a9-26ef-46b8-a1fa-03b1aeac3928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.394022 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435342a9-26ef-46b8-a1fa-03b1aeac3928 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.394126 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1166a0c-dbe0-47b9-916b-788c3c9034e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.394658 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435342a9-26ef-46b8-a1fa-03b1aeac3928 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:07:47.394770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1166a0c-dbe0-47b9-916b-788c3c9034e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.394961 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435342a9-26ef-46b8-a1fa-03b1aeac3928 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.396591 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1166a0c-dbe0-47b9-916b-788c3c9034e9 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.398552 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435342a9-26ef-46b8-a1fa-03b1aeac3928 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.415297 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 624/1240] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:07:47.415297 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435342a9-26ef-46b8-a1fa-03b1aeac3928 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.422758 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 617/1241] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:07:47 2022] POST /identity/v3/users => generated 295 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:47.423440 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1166a0c-dbe0-47b9-916b-788c3c9034e9 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.430341 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435342a9-26ef-46b8-a1fa-03b1aeac3928 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.432735 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1166a0c-dbe0-47b9-916b-788c3c9034e9 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.435283 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bda55a-e062-40c4-9711-e5918f442595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.436007 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bda55a-e062-40c4-9711-e5918f442595 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.436327 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bda55a-e062-40c4-9711-e5918f442595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.436683 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bda55a-e062-40c4-9711-e5918f442595 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.437413 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bda55a-e062-40c4-9711-e5918f442595 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:07:47.437658 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d36b413-1e95-4c37-8ae8-4aa322a34ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.437783 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bda55a-e062-40c4-9711-e5918f442595 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.438249 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36b413-1e95-4c37-8ae8-4aa322a34ada admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.438539 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36b413-1e95-4c37-8ae8-4aa322a34ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.438816 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36b413-1e95-4c37-8ae8-4aa322a34ada admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.439352 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bda55a-e062-40c4-9711-e5918f442595 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.439471 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36b413-1e95-4c37-8ae8-4aa322a34ada admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.441359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36b413-1e95-4c37-8ae8-4aa322a34ada admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.451889 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 625/1242] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:07:47.453517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bda55a-e062-40c4-9711-e5918f442595 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.463618 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bda55a-e062-40c4-9711-e5918f442595 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.470400 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c530b6e-f1a5-4cd4-b071-d7bacba713f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.471310 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c530b6e-f1a5-4cd4-b071-d7bacba713f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.471596 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c530b6e-f1a5-4cd4-b071-d7bacba713f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.471906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c530b6e-f1a5-4cd4-b071-d7bacba713f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.472560 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c530b6e-f1a5-4cd4-b071-d7bacba713f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.474458 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c530b6e-f1a5-4cd4-b071-d7bacba713f2 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.477202 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 618/1243] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:07:47 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:47.478455 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249d865-6dfd-4506-a889-3ee66001ba08 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.478747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249d865-6dfd-4506-a889-3ee66001ba08 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.479043 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249d865-6dfd-4506-a889-3ee66001ba08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.496363 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 626/1244] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:07:47 2022] POST /identity/v3/users => generated 291 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:47.497241 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c530b6e-f1a5-4cd4-b071-d7bacba713f2 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.501973 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-7249d865-6dfd-4506-a889-3ee66001ba08 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:47.503290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-7249d865-6dfd-4506-a889-3ee66001ba08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:47.507205 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c530b6e-f1a5-4cd4-b071-d7bacba713f2 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.563954 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 619/1245] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:47.564731 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7249d865-6dfd-4506-a889-3ee66001ba08 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.568176 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e277277-34a1-4c4e-badf-36b21759f5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.568824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e277277-34a1-4c4e-badf-36b21759f5c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.569312 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e277277-34a1-4c4e-badf-36b21759f5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.569595 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e277277-34a1-4c4e-badf-36b21759f5c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.570830 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e277277-34a1-4c4e-badf-36b21759f5c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:07:47.571140 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e277277-34a1-4c4e-badf-36b21759f5c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.572914 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e277277-34a1-4c4e-badf-36b21759f5c1 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.573737 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7249d865-6dfd-4506-a889-3ee66001ba08 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.579010 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e3a50-3983-42a9-b309-cfcfe4aa0761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.579602 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e3a50-3983-42a9-b309-cfcfe4aa0761 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.579881 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e3a50-3983-42a9-b309-cfcfe4aa0761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.580153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e3a50-3983-42a9-b309-cfcfe4aa0761 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.580830 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e3a50-3983-42a9-b309-cfcfe4aa0761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.582390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e3a50-3983-42a9-b309-cfcfe4aa0761 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.593911 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 627/1246] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:07:47.593911 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e277277-34a1-4c4e-badf-36b21759f5c1 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.594239 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 620/1247] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:47.594945 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e3a50-3983-42a9-b309-cfcfe4aa0761 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.600113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e277277-34a1-4c4e-badf-36b21759f5c1 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.604298 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e3a50-3983-42a9-b309-cfcfe4aa0761 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.604976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62eeef5-693b-4f89-bb5a-7606abf1c1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.605651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62eeef5-693b-4f89-bb5a-7606abf1c1e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.605934 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62eeef5-693b-4f89-bb5a-7606abf1c1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.606211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62eeef5-693b-4f89-bb5a-7606abf1c1e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.606833 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62eeef5-693b-4f89-bb5a-7606abf1c1e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.608757 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62eeef5-693b-4f89-bb5a-7606abf1c1e0 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.609377 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105da98-8d10-4c12-bb01-304581658aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.609957 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105da98-8d10-4c12-bb01-304581658aad admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.610236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105da98-8d10-4c12-bb01-304581658aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.610506 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105da98-8d10-4c12-bb01-304581658aad admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.611159 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a105da98-8d10-4c12-bb01-304581658aad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.612751 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a105da98-8d10-4c12-bb01-304581658aad admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.622480 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 621/1248] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:47.623251 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105da98-8d10-4c12-bb01-304581658aad admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.633957 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105da98-8d10-4c12-bb01-304581658aad admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.637254 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 628/1249] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:07:47 2022] POST /identity/v3/projects => generated 403 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:47.638062 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62eeef5-693b-4f89-bb5a-7606abf1c1e0 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.640565 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a677ce-d4e0-4b5b-b86d-5877403be429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.641326 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a677ce-d4e0-4b5b-b86d-5877403be429 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.641651 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a677ce-d4e0-4b5b-b86d-5877403be429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.641980 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a677ce-d4e0-4b5b-b86d-5877403be429 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.642735 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a677ce-d4e0-4b5b-b86d-5877403be429 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.644712 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a677ce-d4e0-4b5b-b86d-5877403be429 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.673286 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 622/1250] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:07:47 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:47.674024 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a677ce-d4e0-4b5b-b86d-5877403be429 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.683311 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:47.683548 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:47.707373 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62eeef5-693b-4f89-bb5a-7606abf1c1e0 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.737790 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a677ce-d4e0-4b5b-b86d-5877403be429 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.742833 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3c109-929c-4c4d-867d-100e37cb02bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.743443 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3c109-929c-4c4d-867d-100e37cb02bc admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.743726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3c109-929c-4c4d-867d-100e37cb02bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.744003 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3c109-929c-4c4d-867d-100e37cb02bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.744679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb3c109-929c-4c4d-867d-100e37cb02bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.746308 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb3c109-929c-4c4d-867d-100e37cb02bc admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.756922 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 623/1251] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:47.757498 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3c109-929c-4c4d-867d-100e37cb02bc admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.767286 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3c109-929c-4c4d-867d-100e37cb02bc admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.768550 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799515ed-e8ea-497e-93ac-0b633ae582ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.769167 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799515ed-e8ea-497e-93ac-0b633ae582ec admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.769450 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799515ed-e8ea-497e-93ac-0b633ae582ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.769726 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799515ed-e8ea-497e-93ac-0b633ae582ec admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.770350 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799515ed-e8ea-497e-93ac-0b633ae582ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.772058 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf04992-97cc-46c4-ae90-f65264e5cae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.772239 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799515ed-e8ea-497e-93ac-0b633ae582ec admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.772683 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf04992-97cc-46c4-ae90-f65264e5cae0 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.772971 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf04992-97cc-46c4-ae90-f65264e5cae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.773246 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf04992-97cc-46c4-ae90-f65264e5cae0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.773887 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf04992-97cc-46c4-ae90-f65264e5cae0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:07:47.774194 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf04992-97cc-46c4-ae90-f65264e5cae0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.775752 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf04992-97cc-46c4-ae90-f65264e5cae0 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.788509 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 624/1252] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:07:47.789505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf04992-97cc-46c4-ae90-f65264e5cae0 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.795121 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 629/1253] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:07:47 2022] POST /identity/v3/users => generated 284 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:47.796026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799515ed-e8ea-497e-93ac-0b633ae582ec admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.800220 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf04992-97cc-46c4-ae90-f65264e5cae0 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.805601 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d319bf-3fb5-4300-bea7-83ffb347ce2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.806201 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799515ed-e8ea-497e-93ac-0b633ae582ec admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.806337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d319bf-3fb5-4300-bea7-83ffb347ce2f admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.806480 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d319bf-3fb5-4300-bea7-83ffb347ce2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.806753 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d319bf-3fb5-4300-bea7-83ffb347ce2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.807382 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d319bf-3fb5-4300-bea7-83ffb347ce2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.809363 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d319bf-3fb5-4300-bea7-83ffb347ce2f admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.810775 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a199f-a142-4182-af93-86893638ef10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.811846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a199f-a142-4182-af93-86893638ef10 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.812673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a199f-a142-4182-af93-86893638ef10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.812937 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a199f-a142-4182-af93-86893638ef10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.813574 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a199f-a142-4182-af93-86893638ef10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.815125 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a199f-a142-4182-af93-86893638ef10 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.824799 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 630/1254] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:47.825657 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a199f-a142-4182-af93-86893638ef10 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.836665 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a199f-a142-4182-af93-86893638ef10 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.841584 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df3d85-99bc-40f5-9dad-84757a6fa8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.845184 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 625/1255] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:07:47 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:47.845351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df3d85-99bc-40f5-9dad-84757a6fa8fa admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.845351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df3d85-99bc-40f5-9dad-84757a6fa8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.845351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df3d85-99bc-40f5-9dad-84757a6fa8fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.845618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d319bf-3fb5-4300-bea7-83ffb347ce2f admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.845722 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df3d85-99bc-40f5-9dad-84757a6fa8fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.845722 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df3d85-99bc-40f5-9dad-84757a6fa8fa admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.856805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 631/1256] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:47.857661 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df3d85-99bc-40f5-9dad-84757a6fa8fa admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.868196 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df3d85-99bc-40f5-9dad-84757a6fa8fa admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.907686 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d319bf-3fb5-4300-bea7-83ffb347ce2f admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.912341 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f9b27-c5c7-444f-83aa-ab1c96026757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.912975 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f9b27-c5c7-444f-83aa-ab1c96026757 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.913257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f9b27-c5c7-444f-83aa-ab1c96026757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.913534 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f9b27-c5c7-444f-83aa-ab1c96026757 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.914161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549f9b27-c5c7-444f-83aa-ab1c96026757 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.916104 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549f9b27-c5c7-444f-83aa-ab1c96026757 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.924024 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6295d-aab1-457f-8ffa-ce879cb1a871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.924589 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6295d-aab1-457f-8ffa-ce879cb1a871 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.925051 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6295d-aab1-457f-8ffa-ce879cb1a871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.925332 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6295d-aab1-457f-8ffa-ce879cb1a871 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.925958 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe6295d-aab1-457f-8ffa-ce879cb1a871 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.927828 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe6295d-aab1-457f-8ffa-ce879cb1a871 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.938103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 626/1257] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:07:47 2022] POST /identity/v3/users => generated 302 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:47.939786 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f9b27-c5c7-444f-83aa-ab1c96026757 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.951850 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 632/1258] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:07:47 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:47.951850 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6295d-aab1-457f-8ffa-ce879cb1a871 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.952674 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f9b27-c5c7-444f-83aa-ab1c96026757 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.956470 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c109e6-25f5-42b2-853c-47661b8efbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.957173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c109e6-25f5-42b2-853c-47661b8efbf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.957486 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c109e6-25f5-42b2-853c-47661b8efbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.957763 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c109e6-25f5-42b2-853c-47661b8efbf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.958400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c109e6-25f5-42b2-853c-47661b8efbf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.959925 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6295d-aab1-457f-8ffa-ce879cb1a871 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.960038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c109e6-25f5-42b2-853c-47661b8efbf6 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.964409 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5f0d2-bb4b-41b3-a9cd-9a09dc430ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.965020 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5f0d2-bb4b-41b3-a9cd-9a09dc430ac9 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.965813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5f0d2-bb4b-41b3-a9cd-9a09dc430ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.965813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5f0d2-bb4b-41b3-a9cd-9a09dc430ac9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.966425 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b5f0d2-bb4b-41b3-a9cd-9a09dc430ac9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.969119 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b5f0d2-bb4b-41b3-a9cd-9a09dc430ac9 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.971042 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 627/1259] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:47.971690 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c109e6-25f5-42b2-853c-47661b8efbf6 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:47.980958 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c109e6-25f5-42b2-853c-47661b8efbf6 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:47.985590 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9bb5c3-f728-4d4b-a97b-d4c996ff8d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:47.986171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9bb5c3-f728-4d4b-a97b-d4c996ff8d65 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:47.986560 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9bb5c3-f728-4d4b-a97b-d4c996ff8d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:47.986755 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9bb5c3-f728-4d4b-a97b-d4c996ff8d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:47.987398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9bb5c3-f728-4d4b-a97b-d4c996ff8d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:47.989104 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9bb5c3-f728-4d4b-a97b-d4c996ff8d65 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:47.996950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 633/1260] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:07:47 2022] POST /identity/v3/projects => generated 393 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:47.997838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5f0d2-bb4b-41b3-a9cd-9a09dc430ac9 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:48.000273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 628/1261] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:48.001086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9bb5c3-f728-4d4b-a97b-d4c996ff8d65 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:48.010576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9bb5c3-f728-4d4b-a97b-d4c996ff8d65 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:48.023653 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:48.047398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:48.077120 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5f0d2-bb4b-41b3-a9cd-9a09dc430ac9 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:48.079691 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a32c89-160b-433b-9e26-0d142e40d5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:48.080472 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a32c89-160b-433b-9e26-0d142e40d5c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:48.080791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a32c89-160b-433b-9e26-0d142e40d5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:48.081072 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a32c89-160b-433b-9e26-0d142e40d5c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:48.081708 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a32c89-160b-433b-9e26-0d142e40d5c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:48.082548 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bcbfc5-44ea-4d5e-ab1b-6a89f9833a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:48.083209 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bcbfc5-44ea-4d5e-ab1b-6a89f9833a2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:48.083489 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bcbfc5-44ea-4d5e-ab1b-6a89f9833a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:48.083658 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a32c89-160b-433b-9e26-0d142e40d5c0 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:48.083774 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bcbfc5-44ea-4d5e-ab1b-6a89f9833a2d admin admin] PATH_INFO: `/v3/projects/8cfd193661244940b517de9a812796f2/users/796f436ceda84b26a07b7763d57899d0/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:48.099982 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bcbfc5-44ea-4d5e-ab1b-6a89f9833a2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cfd193661244940b517de9a812796f2, user_id=796f436ceda84b26a07b7763d57899d0, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:48.104405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bcbfc5-44ea-4d5e-ab1b-6a89f9833a2d admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:48.106364 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 629/1262] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:07:48 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:48.107192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a32c89-160b-433b-9e26-0d142e40d5c0 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:48.119079 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a32c89-160b-433b-9e26-0d142e40d5c0 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:48.122770 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 634/1263] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:07:47 2022] PUT /identity/v3/projects/8cfd193661244940b517de9a812796f2/users/796f436ceda84b26a07b7763d57899d0/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:07:48.129427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bcbfc5-44ea-4d5e-ab1b-6a89f9833a2d admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:48.177574 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c3bd2-1d06-4f74-abe3-ec43c116b2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:48.178187 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c3bd2-1d06-4f74-abe3-ec43c116b2ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:48.178470 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c3bd2-1d06-4f74-abe3-ec43c116b2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:48.178754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c3bd2-1d06-4f74-abe3-ec43c116b2ca admin admin] PATH_INFO: `/v3/projects/55445c398b424025b2588386af037755/users/8b4ef165353f4f1189400c005519c2ac/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:48.192147 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bcbfc5-44ea-4d5e-ab1b-6a89f9833a2d admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:48.194102 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c3bd2-1d06-4f74-abe3-ec43c116b2ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=55445c398b424025b2588386af037755, user_id=8b4ef165353f4f1189400c005519c2ac, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:48.196763 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4edfe-49d1-4b8c-a243-3245c9e6dfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:48.196924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c3bd2-1d06-4f74-abe3-ec43c116b2ca admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:48.197376 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4edfe-49d1-4b8c-a243-3245c9e6dfa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:48.197663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4edfe-49d1-4b8c-a243-3245c9e6dfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:48.197939 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a4edfe-49d1-4b8c-a243-3245c9e6dfa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:48.198577 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4edfe-49d1-4b8c-a243-3245c9e6dfa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:48.200144 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a4edfe-49d1-4b8c-a243-3245c9e6dfa1 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:48.210264 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 630/1264] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:07:48 2022] PUT /identity/v3/projects/55445c398b424025b2588386af037755/users/8b4ef165353f4f1189400c005519c2ac/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:07:48.211119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c3bd2-1d06-4f74-abe3-ec43c116b2ca admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:48.214198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 635/1265] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:48.215051 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4edfe-49d1-4b8c-a243-3245c9e6dfa1 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:48.225558 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a4edfe-49d1-4b8c-a243-3245c9e6dfa1 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:48.275698 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c3bd2-1d06-4f74-abe3-ec43c116b2ca admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:48.281539 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e995d6a-6963-4621-b2b8-100907dc3518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:48.282144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e995d6a-6963-4621-b2b8-100907dc3518 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:48.282429 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e995d6a-6963-4621-b2b8-100907dc3518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:48.282746 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e995d6a-6963-4621-b2b8-100907dc3518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:48.283345 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e995d6a-6963-4621-b2b8-100907dc3518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:48.283470 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a0645-1fd5-43e6-8992-0c8795a7aa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:48.284014 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a0645-1fd5-43e6-8992-0c8795a7aa77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:48.284299 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a0645-1fd5-43e6-8992-0c8795a7aa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:48.284613 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a0645-1fd5-43e6-8992-0c8795a7aa77 admin admin] PATH_INFO: `/v3/projects/6b3e157518b641c1a4f361add48bca80/users/ea5a0f5f9d024d299b5a867681aa474b/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:48.284975 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e995d6a-6963-4621-b2b8-100907dc3518 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:48.294704 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 631/1266] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:48.295581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e995d6a-6963-4621-b2b8-100907dc3518 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:48.299552 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a0645-1fd5-43e6-8992-0c8795a7aa77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b3e157518b641c1a4f361add48bca80, user_id=ea5a0f5f9d024d299b5a867681aa474b, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:48.302245 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a0645-1fd5-43e6-8992-0c8795a7aa77 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:48.305425 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e995d6a-6963-4621-b2b8-100907dc3518 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:48.310127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d7cfa-058f-454d-9cff-3703eb0f6a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:48.310717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d7cfa-058f-454d-9cff-3703eb0f6a8c admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:48.311002 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d7cfa-058f-454d-9cff-3703eb0f6a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:48.311284 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6d7cfa-058f-454d-9cff-3703eb0f6a8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:48.312694 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d7cfa-058f-454d-9cff-3703eb0f6a8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:48.314273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6d7cfa-058f-454d-9cff-3703eb0f6a8c admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:48.316410 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 636/1267] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:07:48 2022] PUT /identity/v3/projects/6b3e157518b641c1a4f361add48bca80/users/ea5a0f5f9d024d299b5a867681aa474b/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:07:48.317290 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a0645-1fd5-43e6-8992-0c8795a7aa77 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:48.327709 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 632/1268] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:48.327709 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d7cfa-058f-454d-9cff-3703eb0f6a8c admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:48.335381 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6d7cfa-058f-454d-9cff-3703eb0f6a8c admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:48.346944 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:48.370371 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:48.389872 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a0645-1fd5-43e6-8992-0c8795a7aa77 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:48.394798 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6502a-fa7e-43a6-b63a-28e9f41a6cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:48.395399 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6502a-fa7e-43a6-b63a-28e9f41a6cd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:48.395682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6502a-fa7e-43a6-b63a-28e9f41a6cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:48.395958 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6502a-fa7e-43a6-b63a-28e9f41a6cd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:48.396626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6502a-fa7e-43a6-b63a-28e9f41a6cd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:48.398579 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6502a-fa7e-43a6-b63a-28e9f41a6cd4 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:48.398912 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb343c-3127-460d-8acb-62bf68635813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:48.399523 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb343c-3127-460d-8acb-62bf68635813 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:48.399810 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb343c-3127-460d-8acb-62bf68635813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:48.400090 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb343c-3127-460d-8acb-62bf68635813 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:48.400773 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb343c-3127-460d-8acb-62bf68635813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:48.402352 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbb343c-3127-460d-8acb-62bf68635813 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:48.413588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 633/1269] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:48.417990 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb02b42-7264-403a-84ff-1828b06f0100 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:48.418292 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb02b42-7264-403a-84ff-1828b06f0100 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:48.418575 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb02b42-7264-403a-84ff-1828b06f0100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:48.422154 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 637/1270] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:07:48 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:48.422913 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6502a-fa7e-43a6-b63a-28e9f41a6cd4 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:48.432535 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6502a-fa7e-43a6-b63a-28e9f41a6cd4 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:48.437565 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba460be-a8f0-4102-9ff7-663c15f2d37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:48.438271 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba460be-a8f0-4102-9ff7-663c15f2d37a admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:48.438698 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba460be-a8f0-4102-9ff7-663c15f2d37a admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:48.439178 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba460be-a8f0-4102-9ff7-663c15f2d37a admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:48.439969 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba460be-a8f0-4102-9ff7-663c15f2d37a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:48.441674 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba460be-a8f0-4102-9ff7-663c15f2d37a admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:48.448990 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-cdb02b42-7264-403a-84ff-1828b06f0100 None None] MFA Rules not processed for user `796f436ceda84b26a07b7763d57899d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:48.450205 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-cdb02b42-7264-403a-84ff-1828b06f0100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:48.451746 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 638/1271] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:48.453051 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb750a2b-cf09-4464-8bd6-f94d3bdfa9df None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:48.453348 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb750a2b-cf09-4464-8bd6-f94d3bdfa9df None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:48.453630 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb750a2b-cf09-4464-8bd6-f94d3bdfa9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:48.474293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-fb750a2b-cf09-4464-8bd6-f94d3bdfa9df None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:48.475529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-fb750a2b-cf09-4464-8bd6-f94d3bdfa9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:48.484130 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 639/1272] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:48.485365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb750a2b-cf09-4464-8bd6-f94d3bdfa9df None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:48.496566 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb750a2b-cf09-4464-8bd6-f94d3bdfa9df None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:48.501757 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ac0d1-57e1-498b-ad06-2fae310c961c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:48.502359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ac0d1-57e1-498b-ad06-2fae310c961c admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:48.502641 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ac0d1-57e1-498b-ad06-2fae310c961c admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:48.502917 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ac0d1-57e1-498b-ad06-2fae310c961c admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:48.503563 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ac0d1-57e1-498b-ad06-2fae310c961c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:48.505316 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ac0d1-57e1-498b-ad06-2fae310c961c admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:48.515227 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 640/1273] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:48.516588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647ee3f-b312-4755-8591-1fe4a9bc122d None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:48.516881 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647ee3f-b312-4755-8591-1fe4a9bc122d None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:48.517161 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647ee3f-b312-4755-8591-1fe4a9bc122d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:48.518338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 634/1274] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:48 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:48.519369 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb02b42-7264-403a-84ff-1828b06f0100 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:48.531562 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb02b42-7264-403a-84ff-1828b06f0100 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:48.536282 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031f2c0-e25a-4e8a-9575-013ced9c5298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:48.537053 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031f2c0-e25a-4e8a-9575-013ced9c5298 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:48.537339 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031f2c0-e25a-4e8a-9575-013ced9c5298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:48.537626 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031f2c0-e25a-4e8a-9575-013ced9c5298 admin admin] PATH_INFO: `/v3/projects/441c14c2698c4439b9d90f7e2ecfc1ad/users/d029a0e121574898b10942d80212db3b/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:48.548307 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-f647ee3f-b312-4755-8591-1fe4a9bc122d None None] MFA Rules not processed for user `ea5a0f5f9d024d299b5a867681aa474b`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:48.549548 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f647ee3f-b312-4755-8591-1fe4a9bc122d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:48.552930 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b031f2c0-e25a-4e8a-9575-013ced9c5298 admin admin] RBAC: Authorizing `identity:create_grant(project_id=441c14c2698c4439b9d90f7e2ecfc1ad, user_id=d029a0e121574898b10942d80212db3b, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:48.555728 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b031f2c0-e25a-4e8a-9575-013ced9c5298 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:48.568955 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 635/1275] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:07:48 2022] PUT /identity/v3/projects/441c14c2698c4439b9d90f7e2ecfc1ad/users/d029a0e121574898b10942d80212db3b/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:07:48.569770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031f2c0-e25a-4e8a-9575-013ced9c5298 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:48.618485 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 641/1276] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:48.619145 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647ee3f-b312-4755-8591-1fe4a9bc122d None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:48.629506 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647ee3f-b312-4755-8591-1fe4a9bc122d None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:48.633405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031f2c0-e25a-4e8a-9575-013ced9c5298 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:48.634066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2906ff14-8ecd-417a-81e5-7caa3b623974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:48.634655 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906ff14-8ecd-417a-81e5-7caa3b623974 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:48.634940 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906ff14-8ecd-417a-81e5-7caa3b623974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:48.635217 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2906ff14-8ecd-417a-81e5-7caa3b623974 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:48.635857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2906ff14-8ecd-417a-81e5-7caa3b623974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:48.637463 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2906ff14-8ecd-417a-81e5-7caa3b623974 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:48.638491 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe25917-8bac-4a67-9266-f36f9feba57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:48.639099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe25917-8bac-4a67-9266-f36f9feba57c admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:48.639386 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe25917-8bac-4a67-9266-f36f9feba57c admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:48.640166 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe25917-8bac-4a67-9266-f36f9feba57c admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:48.640998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe25917-8bac-4a67-9266-f36f9feba57c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:48.643104 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe25917-8bac-4a67-9266-f36f9feba57c admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:48.648176 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 642/1277] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:48.649063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2906ff14-8ecd-417a-81e5-7caa3b623974 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:48.653255 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 636/1278] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:48.653949 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe25917-8bac-4a67-9266-f36f9feba57c admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:48.660009 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2906ff14-8ecd-417a-81e5-7caa3b623974 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:48.664953 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c0bfb-003d-42af-891e-c5668be137b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:48.665600 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c0bfb-003d-42af-891e-c5668be137b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:48.665885 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c0bfb-003d-42af-891e-c5668be137b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:48.666186 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c0bfb-003d-42af-891e-c5668be137b5 admin admin] PATH_INFO: `/v3/projects/7cd8dc6fdb244b24bdaf951caabc10e2/users/a3ea1c4cb0894cc7a6202caeffa38bc9/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:48.681156 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c0bfb-003d-42af-891e-c5668be137b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cd8dc6fdb244b24bdaf951caabc10e2, user_id=a3ea1c4cb0894cc7a6202caeffa38bc9, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:48.683830 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c0bfb-003d-42af-891e-c5668be137b5 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:48.690470 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:48.699632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 643/1279] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:07:48 2022] PUT /identity/v3/projects/7cd8dc6fdb244b24bdaf951caabc10e2/users/a3ea1c4cb0894cc7a6202caeffa38bc9/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:07:48.700308 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c0bfb-003d-42af-891e-c5668be137b5 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:48.717376 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe25917-8bac-4a67-9266-f36f9feba57c admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:48.740392 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:48.764202 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c0bfb-003d-42af-891e-c5668be137b5 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:48.767692 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d96752-948a-40dd-971a-b2a7c57d2eb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:48.768541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d96752-948a-40dd-971a-b2a7c57d2eb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:48.768923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d96752-948a-40dd-971a-b2a7c57d2eb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:48.769268 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d96752-948a-40dd-971a-b2a7c57d2eb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:48.769491 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40670d5-78ff-4cda-9b93-9ca5527653bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:48.770346 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40670d5-78ff-4cda-9b93-9ca5527653bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:48.770717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40670d5-78ff-4cda-9b93-9ca5527653bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:48.771092 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40670d5-78ff-4cda-9b93-9ca5527653bb admin admin] PATH_INFO: `/v3/projects/06566ec2efab47ea80f2f10723b478a5/users/127eff42dfb141c3b78ce04636af1f28/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:48.778824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d96752-948a-40dd-971a-b2a7c57d2eb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:48.780984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d96752-948a-40dd-971a-b2a7c57d2eb5 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:48.786141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40670d5-78ff-4cda-9b93-9ca5527653bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=06566ec2efab47ea80f2f10723b478a5, user_id=127eff42dfb141c3b78ce04636af1f28, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:48.788782 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40670d5-78ff-4cda-9b93-9ca5527653bb admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:48.790846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 637/1280] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:07:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:07:48.791528 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d96752-948a-40dd-971a-b2a7c57d2eb5 service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:48.800762 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d96752-948a-40dd-971a-b2a7c57d2eb5 service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:48.803007 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 644/1281] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:07:48 2022] PUT /identity/v3/projects/06566ec2efab47ea80f2f10723b478a5/users/127eff42dfb141c3b78ce04636af1f28/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:07:48.804841 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5021a32-484e-4a14-90b1-a191a9fb0858 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:48.805139 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5021a32-484e-4a14-90b1-a191a9fb0858 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:48.805960 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5021a32-484e-4a14-90b1-a191a9fb0858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:48.841760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-c5021a32-484e-4a14-90b1-a191a9fb0858 None None] MFA Rules not processed for user `d029a0e121574898b10942d80212db3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:48.843820 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-c5021a32-484e-4a14-90b1-a191a9fb0858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:48.861280 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b952c2-02c3-4c22-8018-cea8efb5a36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:48.861898 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b952c2-02c3-4c22-8018-cea8efb5a36e admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:48.862181 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b952c2-02c3-4c22-8018-cea8efb5a36e admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:48.862460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b952c2-02c3-4c22-8018-cea8efb5a36e admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:48.863544 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b952c2-02c3-4c22-8018-cea8efb5a36e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:48.866081 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b952c2-02c3-4c22-8018-cea8efb5a36e admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:48.897820 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 638/1282] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:07:48 2022] POST /identity/v3/projects => generated 403 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:48.897820 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b952c2-02c3-4c22-8018-cea8efb5a36e admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:48.919270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 645/1283] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:48.923318 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5021a32-484e-4a14-90b1-a191a9fb0858 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:48.936300 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5021a32-484e-4a14-90b1-a191a9fb0858 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:48.973992 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b952c2-02c3-4c22-8018-cea8efb5a36e admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:48.978388 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16073693-0705-4606-94dd-1092dd0f07d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:48.979005 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16073693-0705-4606-94dd-1092dd0f07d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:48.979293 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16073693-0705-4606-94dd-1092dd0f07d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:48.979578 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16073693-0705-4606-94dd-1092dd0f07d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:48.980212 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16073693-0705-4606-94dd-1092dd0f07d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:48.982342 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16073693-0705-4606-94dd-1092dd0f07d8 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:49.006940 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e418d7-fbd3-4113-a689-74a4ca4c5db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:49.007664 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e418d7-fbd3-4113-a689-74a4ca4c5db3 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:49.007855 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e418d7-fbd3-4113-a689-74a4ca4c5db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:49.008266 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e418d7-fbd3-4113-a689-74a4ca4c5db3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:49.008816 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e418d7-fbd3-4113-a689-74a4ca4c5db3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:49.010442 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e418d7-fbd3-4113-a689-74a4ca4c5db3 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:49.015268 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 639/1284] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:07:48 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:49.015268 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16073693-0705-4606-94dd-1092dd0f07d8 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:49.020251 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:49.022120 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 646/1285] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:49.028771 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca9561-a996-42b5-947c-d7b2f7fa086c None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:49.028771 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca9561-a996-42b5-947c-d7b2f7fa086c None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:49.029047 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca9561-a996-42b5-947c-d7b2f7fa086c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:49.056025 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-7cca9561-a996-42b5-947c-d7b2f7fa086c None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:49.057610 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-7cca9561-a996-42b5-947c-d7b2f7fa086c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:49.093324 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16073693-0705-4606-94dd-1092dd0f07d8 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:49.098432 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a8858-d2da-4836-8d32-c0182f14c7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:49.099550 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a8858-d2da-4836-8d32-c0182f14c7f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:49.099843 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a8858-d2da-4836-8d32-c0182f14c7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:49.100134 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a8858-d2da-4836-8d32-c0182f14c7f8 admin admin] PATH_INFO: `/v3/projects/6d730bc61aaa4ef88dc3961fe3f498e8/users/40e1a487cab9418480172aa1f32d7caf/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:49.113681 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:49.127762 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 647/1286] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:49.128640 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a8858-d2da-4836-8d32-c0182f14c7f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d730bc61aaa4ef88dc3961fe3f498e8, user_id=40e1a487cab9418480172aa1f32d7caf, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:49.129114 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6ae00-b900-4617-b82e-5e8200c3b799 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:49.129412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6ae00-b900-4617-b82e-5e8200c3b799 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:49.129690 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6ae00-b900-4617-b82e-5e8200c3b799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:49.132736 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a8858-d2da-4836-8d32-c0182f14c7f8 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:49.149367 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 640/1287] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:07:49 2022] PUT /identity/v3/projects/6d730bc61aaa4ef88dc3961fe3f498e8/users/40e1a487cab9418480172aa1f32d7caf/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:07:49.150473 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a8858-d2da-4836-8d32-c0182f14c7f8 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:49.165805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-28d6ae00-b900-4617-b82e-5e8200c3b799 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:49.167115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-28d6ae00-b900-4617-b82e-5e8200c3b799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:49.227246 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a8858-d2da-4836-8d32-c0182f14c7f8 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:49.231826 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc61cdd-f133-4b7a-95c4-46a05f4460e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:49.232557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc61cdd-f133-4b7a-95c4-46a05f4460e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:49.232911 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc61cdd-f133-4b7a-95c4-46a05f4460e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:49.233284 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc61cdd-f133-4b7a-95c4-46a05f4460e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:49.234028 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc61cdd-f133-4b7a-95c4-46a05f4460e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:49.235950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 648/1288] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:49.236091 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc61cdd-f133-4b7a-95c4-46a05f4460e3 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:49.237383 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6ae00-b900-4617-b82e-5e8200c3b799 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:49.263817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 641/1289] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:07:49 2022] POST /identity/v3/users => generated 308 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:49.263817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc61cdd-f133-4b7a-95c4-46a05f4460e3 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:49.283401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc61cdd-f133-4b7a-95c4-46a05f4460e3 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:49.294020 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d692b71-7227-4113-90fc-82fcc658198f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:49.294671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d692b71-7227-4113-90fc-82fcc658198f admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:49.294953 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d692b71-7227-4113-90fc-82fcc658198f admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:49.295455 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d692b71-7227-4113-90fc-82fcc658198f admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:49.296895 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d692b71-7227-4113-90fc-82fcc658198f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:49.298861 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d692b71-7227-4113-90fc-82fcc658198f admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:49.321197 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 642/1290] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:07:49 2022] POST /identity/v3/users => generated 296 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:49.322010 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d692b71-7227-4113-90fc-82fcc658198f admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:49.326713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6ae00-b900-4617-b82e-5e8200c3b799 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:49.331327 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac542b5-c338-40a3-b3c2-ad454813c131 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:49.331940 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac542b5-c338-40a3-b3c2-ad454813c131 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:49.332227 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac542b5-c338-40a3-b3c2-ad454813c131 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:49.332542 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac542b5-c338-40a3-b3c2-ad454813c131 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:49.334876 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d692b71-7227-4113-90fc-82fcc658198f admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:49.340370 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d2085-3749-4e84-8a7b-a33647ea2371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:49.341487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d2085-3749-4e84-8a7b-a33647ea2371 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:49.341902 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d2085-3749-4e84-8a7b-a33647ea2371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:49.342158 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d2085-3749-4e84-8a7b-a33647ea2371 admin admin] PATH_INFO: `/v3/projects/08beca89c7f747229a3cc6cb6a363af4/users/f95f4fc0d5c5428c96545f543139489b/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:49.361094 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d2085-3749-4e84-8a7b-a33647ea2371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08beca89c7f747229a3cc6cb6a363af4, user_id=f95f4fc0d5c5428c96545f543139489b, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:49.364193 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91d2085-3749-4e84-8a7b-a33647ea2371 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:49.385113 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 643/1291] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:07:49 2022] PUT /identity/v3/projects/08beca89c7f747229a3cc6cb6a363af4/users/f95f4fc0d5c5428c96545f543139489b/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:07:49.386118 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d2085-3749-4e84-8a7b-a33647ea2371 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:49.388883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f91d2085-3749-4e84-8a7b-a33647ea2371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:49.406121 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac542b5-c338-40a3-b3c2-ad454813c131 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:49.407784 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac542b5-c338-40a3-b3c2-ad454813c131 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:49.420768 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 649/1292] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:07:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:07:49.422515 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f7424-ea7e-4d9c-91f5-a6521d358650 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:49.422927 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f7424-ea7e-4d9c-91f5-a6521d358650 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:49.423272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f7424-ea7e-4d9c-91f5-a6521d358650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:49.439481 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:49.459258 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-ad2f7424-ea7e-4d9c-91f5-a6521d358650 None None] MFA Rules not processed for user `40e1a487cab9418480172aa1f32d7caf`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:49.460983 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ad2f7424-ea7e-4d9c-91f5-a6521d358650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:49.467887 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d2085-3749-4e84-8a7b-a33647ea2371 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:49.472090 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882274c1-a146-4c3c-9b10-e3e117717d29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:49.473200 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882274c1-a146-4c3c-9b10-e3e117717d29 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:49.473471 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882274c1-a146-4c3c-9b10-e3e117717d29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:49.473781 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882274c1-a146-4c3c-9b10-e3e117717d29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:49.497348 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:49.540467 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 650/1293] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:49.541739 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f7424-ea7e-4d9c-91f5-a6521d358650 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:49.556054 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f7424-ea7e-4d9c-91f5-a6521d358650 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:49.560222 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ab045-daed-41ba-9ddb-73ab5d05914f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:49.561105 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ab045-daed-41ba-9ddb-73ab5d05914f service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:49.561602 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ab045-daed-41ba-9ddb-73ab5d05914f service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:49.561942 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ab045-daed-41ba-9ddb-73ab5d05914f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:49.567543 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882274c1-a146-4c3c-9b10-e3e117717d29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:49.569426 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882274c1-a146-4c3c-9b10-e3e117717d29 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:49.575343 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ab045-daed-41ba-9ddb-73ab5d05914f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:49.576903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7ab045-daed-41ba-9ddb-73ab5d05914f service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:49.582290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 644/1294] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:07:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:07:49.583387 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882274c1-a146-4c3c-9b10-e3e117717d29 service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:49.589194 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 651/1295] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:07:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:07:49.590054 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ab045-daed-41ba-9ddb-73ab5d05914f service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:49.596789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882274c1-a146-4c3c-9b10-e3e117717d29 service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:49.602467 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e948cb-30b8-4f6b-88ff-196d6616731e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:49.602948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ab045-daed-41ba-9ddb-73ab5d05914f service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:49.603385 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e948cb-30b8-4f6b-88ff-196d6616731e admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:49.603814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e948cb-30b8-4f6b-88ff-196d6616731e admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:49.604213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e948cb-30b8-4f6b-88ff-196d6616731e admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:49.605203 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e948cb-30b8-4f6b-88ff-196d6616731e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:49.607535 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e948cb-30b8-4f6b-88ff-196d6616731e admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:49.608621 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf7304-d5fb-4c06-bfc1-96af1234f71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:49.609161 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf7304-d5fb-4c06-bfc1-96af1234f71a admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:49.609443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf7304-d5fb-4c06-bfc1-96af1234f71a admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:49.609721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf7304-d5fb-4c06-bfc1-96af1234f71a admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:49.610599 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf7304-d5fb-4c06-bfc1-96af1234f71a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:49.612269 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf7304-d5fb-4c06-bfc1-96af1234f71a admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:49.622184 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 645/1296] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:49.626735 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84558a-62a3-4a9f-b790-5cc75db0686a None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:49.626900 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 652/1297] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:49.627003 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84558a-62a3-4a9f-b790-5cc75db0686a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:49.627003 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84558a-62a3-4a9f-b790-5cc75db0686a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:49.627195 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3a811-5da0-47d8-93e1-def9caaf721b None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:49.627195 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3a811-5da0-47d8-93e1-def9caaf721b None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:49.627195 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3a811-5da0-47d8-93e1-def9caaf721b None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:49.627848 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 653/1298] 10.222.0.13 () {54 vars in 961 bytes} [Sun Oct 23 00:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:07:49.629124 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3a811-5da0-47d8-93e1-def9caaf721b None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:49.643653 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3a811-5da0-47d8-93e1-def9caaf721b None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:49.648936 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58760470-7cd6-4dab-b3d8-b3b2b6650440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:49.651566 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58760470-7cd6-4dab-b3d8-b3b2b6650440 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:49.651566 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58760470-7cd6-4dab-b3d8-b3b2b6650440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:49.651566 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58760470-7cd6-4dab-b3d8-b3b2b6650440 admin admin] PATH_INFO: `/v3/projects/21c88da0e7524112bee63371333ddd8d/users/7ecc4a3304a84bd286a79cf0f4178887/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:49.655804 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-0e84558a-62a3-4a9f-b790-5cc75db0686a None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:49.658934 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-0e84558a-62a3-4a9f-b790-5cc75db0686a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:49.674578 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 646/1299] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:49.675241 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58760470-7cd6-4dab-b3d8-b3b2b6650440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21c88da0e7524112bee63371333ddd8d, user_id=7ecc4a3304a84bd286a79cf0f4178887, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:49.676417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84558a-62a3-4a9f-b790-5cc75db0686a None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:49.678219 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58760470-7cd6-4dab-b3d8-b3b2b6650440 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:49.693180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84558a-62a3-4a9f-b790-5cc75db0686a None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:49.704549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 654/1300] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:07:49 2022] PUT /identity/v3/projects/21c88da0e7524112bee63371333ddd8d/users/7ecc4a3304a84bd286a79cf0f4178887/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:07:49.704549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037a3f6-94fe-4aba-980d-80020dd77e5b None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:49.705736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037a3f6-94fe-4aba-980d-80020dd77e5b None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:49.706036 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2037a3f6-94fe-4aba-980d-80020dd77e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:49.762158 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-2037a3f6-94fe-4aba-980d-80020dd77e5b None None] MFA Rules not processed for user `c4a3579d56294b9d8ad36d4d174c8526`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:49.764059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-2037a3f6-94fe-4aba-980d-80020dd77e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:49.771727 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d97e2a-dc2b-436c-a85a-ad916d03c6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:49.772603 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d97e2a-dc2b-436c-a85a-ad916d03c6f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:49.773597 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d97e2a-dc2b-436c-a85a-ad916d03c6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:49.774018 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d97e2a-dc2b-436c-a85a-ad916d03c6f9 admin admin] PATH_INFO: `/v3/projects/f9389ae2621746bcaee099404ca3ce92/users/713ac404acf343888cf2d44ad16748bc/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:49.797617 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d97e2a-dc2b-436c-a85a-ad916d03c6f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9389ae2621746bcaee099404ca3ce92, user_id=713ac404acf343888cf2d44ad16748bc, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:49.801197 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d97e2a-dc2b-436c-a85a-ad916d03c6f9 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:49.834973 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 647/1301] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:07:49 2022] PUT /identity/v3/projects/f9389ae2621746bcaee099404ca3ce92/users/713ac404acf343888cf2d44ad16748bc/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:07:49.838880 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d97e2a-dc2b-436c-a85a-ad916d03c6f9 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:49.886863 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 655/1302] 10.222.0.13 () {58 vars in 1058 bytes} [Sun Oct 23 00:07:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:49.888446 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037a3f6-94fe-4aba-980d-80020dd77e5b None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:49.903443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2037a3f6-94fe-4aba-980d-80020dd77e5b None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:49.914303 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:49.922076 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d97e2a-dc2b-436c-a85a-ad916d03c6f9 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:49.928224 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d60d5-9ffc-4593-9cac-e56278c0d0ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:49.929341 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d60d5-9ffc-4593-9cac-e56278c0d0ad service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:49.930021 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d60d5-9ffc-4593-9cac-e56278c0d0ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:49.930823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505d60d5-9ffc-4593-9cac-e56278c0d0ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:50.013403 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505d60d5-9ffc-4593-9cac-e56278c0d0ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:50.018928 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505d60d5-9ffc-4593-9cac-e56278c0d0ad service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:50.031622 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 648/1303] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:07:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:07:50.032820 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d60d5-9ffc-4593-9cac-e56278c0d0ad service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:50.033738 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d01bf9-d251-4bfe-9bf1-cd92544beb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:50.037385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d01bf9-d251-4bfe-9bf1-cd92544beb1d admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:50.037906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d01bf9-d251-4bfe-9bf1-cd92544beb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:50.038658 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d01bf9-d251-4bfe-9bf1-cd92544beb1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:50.040265 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d01bf9-d251-4bfe-9bf1-cd92544beb1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:50.044479 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505d60d5-9ffc-4593-9cac-e56278c0d0ad service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:50.045224 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d01bf9-d251-4bfe-9bf1-cd92544beb1d admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:50.050563 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a691c6-6d1b-4b6f-9693-227a70fcd1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:50.051296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a691c6-6d1b-4b6f-9693-227a70fcd1eb admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:50.051883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a691c6-6d1b-4b6f-9693-227a70fcd1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:50.052230 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a691c6-6d1b-4b6f-9693-227a70fcd1eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:50.052871 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:50.053085 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a691c6-6d1b-4b6f-9693-227a70fcd1eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:50.055414 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a691c6-6d1b-4b6f-9693-227a70fcd1eb admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:50.074277 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 649/1304] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:50.076287 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a691c6-6d1b-4b6f-9693-227a70fcd1eb admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:50.096426 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 656/1305] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:50.098749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d01bf9-d251-4bfe-9bf1-cd92544beb1d admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:50.126745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d01bf9-d251-4bfe-9bf1-cd92544beb1d admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:50.141442 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a4381-3bce-4855-9ec6-bb9fe7c1421f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:50.142228 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a4381-3bce-4855-9ec6-bb9fe7c1421f admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:50.142715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a4381-3bce-4855-9ec6-bb9fe7c1421f admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:50.143168 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a4381-3bce-4855-9ec6-bb9fe7c1421f admin admin] PATH_INFO: `/v3/projects/f9389ae2621746bcaee099404ca3ce92/users/713ac404acf343888cf2d44ad16748bc/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:50.147382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a4381-3bce-4855-9ec6-bb9fe7c1421f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9389ae2621746bcaee099404ca3ce92, user_id=713ac404acf343888cf2d44ad16748bc, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:50.150195 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a4381-3bce-4855-9ec6-bb9fe7c1421f admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:50.179089 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 657/1306] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:07:50 2022] PUT /identity/v3/projects/f9389ae2621746bcaee099404ca3ce92/users/713ac404acf343888cf2d44ad16748bc/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:07:50.180121 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a4381-3bce-4855-9ec6-bb9fe7c1421f admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:50.209577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a691c6-6d1b-4b6f-9693-227a70fcd1eb admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:50.408787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a4381-3bce-4855-9ec6-bb9fe7c1421f admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:50.423339 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f37815-3ea7-4e36-84e7-db03c16f9da0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:50.424657 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f37815-3ea7-4e36-84e7-db03c16f9da0 service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:50.424950 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f37815-3ea7-4e36-84e7-db03c16f9da0 service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:50.425234 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f37815-3ea7-4e36-84e7-db03c16f9da0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:50.430752 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65e067-76ca-4867-81a3-ba9b6effe561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:50.431549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65e067-76ca-4867-81a3-ba9b6effe561 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:50.431899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65e067-76ca-4867-81a3-ba9b6effe561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:50.432293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65e067-76ca-4867-81a3-ba9b6effe561 admin admin] PATH_INFO: `/v3/projects/21c88da0e7524112bee63371333ddd8d/users/7ecc4a3304a84bd286a79cf0f4178887/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:50.439863 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65e067-76ca-4867-81a3-ba9b6effe561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21c88da0e7524112bee63371333ddd8d, user_id=7ecc4a3304a84bd286a79cf0f4178887, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:50.444861 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65e067-76ca-4867-81a3-ba9b6effe561 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:50.477480 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:50.486939 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 658/1307] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:07:50 2022] PUT /identity/v3/projects/21c88da0e7524112bee63371333ddd8d/users/7ecc4a3304a84bd286a79cf0f4178887/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:07:50.489179 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd47c6-3c38-44e8-bafa-45f7a6078eb4 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:50.490042 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd47c6-3c38-44e8-bafa-45f7a6078eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:50.490042 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd47c6-3c38-44e8-bafa-45f7a6078eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:50.525598 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-11fd47c6-3c38-44e8-bafa-45f7a6078eb4 None None] MFA Rules not processed for user `713ac404acf343888cf2d44ad16748bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:50.527272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-11fd47c6-3c38-44e8-bafa-45f7a6078eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:50.527592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f37815-3ea7-4e36-84e7-db03c16f9da0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:50.529640 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f37815-3ea7-4e36-84e7-db03c16f9da0 service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:50.540292 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 650/1308] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:07:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:07:50.542171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdf067-c0f2-4f4f-a77d-30bf40ce250e None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:50.542728 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdf067-c0f2-4f4f-a77d-30bf40ce250e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:50.543273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdf067-c0f2-4f4f-a77d-30bf40ce250e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:50.581067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-57cdf067-c0f2-4f4f-a77d-30bf40ce250e None None] MFA Rules not processed for user `7ecc4a3304a84bd286a79cf0f4178887`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:50.583228 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-57cdf067-c0f2-4f4f-a77d-30bf40ce250e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:50.680293 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 651/1309] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:50 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:50.706012 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cdf067-c0f2-4f4f-a77d-30bf40ce250e None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:50.752666 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 659/1310] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:50 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:50.761359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd47c6-3c38-44e8-bafa-45f7a6078eb4 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:50.813553 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cdf067-c0f2-4f4f-a77d-30bf40ce250e None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:50.818244 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29071a76-dc62-45c3-bd95-91a9e11c8e18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:50.819204 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29071a76-dc62-45c3-bd95-91a9e11c8e18 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:50.819660 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29071a76-dc62-45c3-bd95-91a9e11c8e18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:50.820111 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29071a76-dc62-45c3-bd95-91a9e11c8e18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:50.831931 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29071a76-dc62-45c3-bd95-91a9e11c8e18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:50.833703 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29071a76-dc62-45c3-bd95-91a9e11c8e18 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:50.853004 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 652/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:07:50 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:07:50.893838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd47c6-3c38-44e8-bafa-45f7a6078eb4 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:50.908528 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c50aca-2664-453a-b279-2ab9419c815b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:50.909168 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c50aca-2664-453a-b279-2ab9419c815b service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:50.909457 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c50aca-2664-453a-b279-2ab9419c815b service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:50.909734 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c50aca-2664-453a-b279-2ab9419c815b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:50.938387 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c50aca-2664-453a-b279-2ab9419c815b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:50.939258 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c50aca-2664-453a-b279-2ab9419c815b service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:50.970461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 660/1312] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:07:50 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:07:51.115654 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b4935-4f45-4986-902a-29b6372c15c8 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:51.116244 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b4935-4f45-4986-902a-29b6372c15c8 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:51.116616 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b4935-4f45-4986-902a-29b6372c15c8 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:51.118317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 653/1313] 10.222.0.13 () {54 vars in 912 bytes} [Sun Oct 23 00:07:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:07:51.123959 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b54e96-4883-4b76-ab0b-222c9c646375 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:51.124385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b54e96-4883-4b76-ab0b-222c9c646375 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:51.124721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b54e96-4883-4b76-ab0b-222c9c646375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:51.137489 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:51.163136 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b4935-4f45-4986-902a-29b6372c15c8 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:51.169332 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-87b54e96-4883-4b76-ab0b-222c9c646375 None None] MFA Rules not processed for user `c4a3579d56294b9d8ad36d4d174c8526`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:51.171933 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-87b54e96-4883-4b76-ab0b-222c9c646375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:51.241661 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:51.259449 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b4935-4f45-4986-902a-29b6372c15c8 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:51.265941 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aceddfc-a8cf-4a91-8df8-291860147634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:51.267802 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aceddfc-a8cf-4a91-8df8-291860147634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:51.268319 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aceddfc-a8cf-4a91-8df8-291860147634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:51.268678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aceddfc-a8cf-4a91-8df8-291860147634 admin admin] PATH_INFO: `/v3/users/796f436ceda84b26a07b7763d57899d0` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:51.270107 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aceddfc-a8cf-4a91-8df8-291860147634 admin admin] RBAC: Authorizing `identity:delete_user(user_id=796f436ceda84b26a07b7763d57899d0)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:51.273272 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aceddfc-a8cf-4a91-8df8-291860147634 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:51.302163 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 661/1314] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 00:07:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:51.305040 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b54e96-4883-4b76-ab0b-222c9c646375 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:51.320059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b54e96-4883-4b76-ab0b-222c9c646375 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:51.325106 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d26b224-267f-44b1-9a24-5bee651f9085 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:51.325827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26b224-267f-44b1-9a24-5bee651f9085 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:51.326116 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26b224-267f-44b1-9a24-5bee651f9085 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:51.326437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d26b224-267f-44b1-9a24-5bee651f9085 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:51.365064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-9aceddfc-a8cf-4a91-8df8-291860147634 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '796f436ceda84b26a07b7763d57899d0'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:07:51.366625 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-9aceddfc-a8cf-4a91-8df8-291860147634 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '796f436ceda84b26a07b7763d57899d0'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:07:51.389375 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-9aceddfc-a8cf-4a91-8df8-291860147634 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '796f436ceda84b26a07b7763d57899d0'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:07:51.429857 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-9aceddfc-a8cf-4a91-8df8-291860147634 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '796f436ceda84b26a07b7763d57899d0'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:07:51.436898 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d26b224-267f-44b1-9a24-5bee651f9085 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:51.438912 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d26b224-267f-44b1-9a24-5bee651f9085 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:51.445007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-9aceddfc-a8cf-4a91-8df8-291860147634 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '796f436ceda84b26a07b7763d57899d0'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:07:51.466733 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 654/1315] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:07:51 2022] DELETE /identity/v3/users/796f436ceda84b26a07b7763d57899d0 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:07:51.474546 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aceddfc-a8cf-4a91-8df8-291860147634 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:51.479122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9aceddfc-a8cf-4a91-8df8-291860147634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:51.480736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 662/1316] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:07:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:07:51.484819 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03e0e3-37ab-4af5-b126-26b74caf16a8 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:51.485118 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03e0e3-37ab-4af5-b126-26b74caf16a8 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:51.486657 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03e0e3-37ab-4af5-b126-26b74caf16a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:51.533630 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-ee03e0e3-37ab-4af5-b126-26b74caf16a8 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:51.539047 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ee03e0e3-37ab-4af5-b126-26b74caf16a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:51.642035 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aceddfc-a8cf-4a91-8df8-291860147634 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:51.652141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590b965-ef02-4fe1-836b-d191d1f5d7dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:51.652809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590b965-ef02-4fe1-836b-d191d1f5d7dd service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:51.653095 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590b965-ef02-4fe1-836b-d191d1f5d7dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:51.653374 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590b965-ef02-4fe1-836b-d191d1f5d7dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:51.657606 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-7590b965-ef02-4fe1-836b-d191d1f5d7dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:51.680866 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 663/1317] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:51.724638 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03e0e3-37ab-4af5-b126-26b74caf16a8 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:51.727020 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ee03e0e3-37ab-4af5-b126-26b74caf16a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:51.820588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:51.849833 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7590b965-ef02-4fe1-836b-d191d1f5d7dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:51.866380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03e0e3-37ab-4af5-b126-26b74caf16a8 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:51.867295 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7590b965-ef02-4fe1-836b-d191d1f5d7dd service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:51.873667 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8437ba4-bbab-4ff0-a555-7ebf16863381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:51.874534 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8437ba4-bbab-4ff0-a555-7ebf16863381 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:51.874830 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8437ba4-bbab-4ff0-a555-7ebf16863381 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:51.875110 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8437ba4-bbab-4ff0-a555-7ebf16863381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:51.887479 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8437ba4-bbab-4ff0-a555-7ebf16863381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:51.889046 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8437ba4-bbab-4ff0-a555-7ebf16863381 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:51.900912 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 655/1318] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:07:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:07:51.903956 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 664/1319] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:07:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:07:52.364912 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308026f-f8aa-4b1f-aa96-5e874038ea66 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:52.365416 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308026f-f8aa-4b1f-aa96-5e874038ea66 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:52.365869 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308026f-f8aa-4b1f-aa96-5e874038ea66 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:52.367927 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 656/1320] 10.222.0.13 () {54 vars in 961 bytes} [Sun Oct 23 00:07:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:07:52.376028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b4cbd-fe9b-4880-8116-a46639aa4705 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:52.376408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b4cbd-fe9b-4880-8116-a46639aa4705 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:52.377028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b4cbd-fe9b-4880-8116-a46639aa4705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:52.458063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-ad4b4cbd-fe9b-4880-8116-a46639aa4705 None None] MFA Rules not processed for user `c4a3579d56294b9d8ad36d4d174c8526`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:52.460960 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ad4b4cbd-fe9b-4880-8116-a46639aa4705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:52.506779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 665/1321] 10.222.0.13 () {58 vars in 1058 bytes} [Sun Oct 23 00:07:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:52.512132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b78a6-123e-4109-89a3-9430fa28e947 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:52.512888 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b78a6-123e-4109-89a3-9430fa28e947 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:52.513475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b78a6-123e-4109-89a3-9430fa28e947 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:52.516651 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 657/1322] 10.222.0.13 () {54 vars in 912 bytes} [Sun Oct 23 00:07:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:07:52.523015 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef2233-5f23-4ed2-87bf-df9758190fe5 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:52.523015 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef2233-5f23-4ed2-87bf-df9758190fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:52.523015 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef2233-5f23-4ed2-87bf-df9758190fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:52.550697 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-39ef2233-5f23-4ed2-87bf-df9758190fe5 None None] MFA Rules not processed for user `c4a3579d56294b9d8ad36d4d174c8526`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:52.553182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-39ef2233-5f23-4ed2-87bf-df9758190fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:52.569601 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 666/1323] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 00:07:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:52.583476 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b78a6-123e-4109-89a3-9430fa28e947 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:52.585518 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-636b78a6-123e-4109-89a3-9430fa28e947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:52.604868 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b78a6-123e-4109-89a3-9430fa28e947 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:52.620233 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9700bf1-ec7b-40e4-9798-c2c0c08da034 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:52.621078 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9700bf1-ec7b-40e4-9798-c2c0c08da034 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:52.621711 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9700bf1-ec7b-40e4-9798-c2c0c08da034 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:52.622148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9700bf1-ec7b-40e4-9798-c2c0c08da034 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:52.667010 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9700bf1-ec7b-40e4-9798-c2c0c08da034 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:52.670107 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9700bf1-ec7b-40e4-9798-c2c0c08da034 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:52.682888 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:52.693420 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 658/1324] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:07:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:07:52.714952 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061095c8-b9d9-4472-97b1-004cfb13089e None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:52.715937 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061095c8-b9d9-4472-97b1-004cfb13089e None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:52.717543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061095c8-b9d9-4472-97b1-004cfb13089e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:52.769393 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-061095c8-b9d9-4472-97b1-004cfb13089e None None] MFA Rules not processed for user `a3ea1c4cb0894cc7a6202caeffa38bc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:52.770608 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-061095c8-b9d9-4472-97b1-004cfb13089e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:52.918697 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 667/1325] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:52 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:52.924823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9700bf1-ec7b-40e4-9798-c2c0c08da034 service cinder] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:52.926721 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c9700bf1-ec7b-40e4-9798-c2c0c08da034 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:52.945800 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9700bf1-ec7b-40e4-9798-c2c0c08da034 service cinder] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:52.954139 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8a0e4-367b-43d7-a9d3-95887401d9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:52.955191 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8a0e4-367b-43d7-a9d3-95887401d9fb admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:52.955729 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8a0e4-367b-43d7-a9d3-95887401d9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:52.956309 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8a0e4-367b-43d7-a9d3-95887401d9fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:52.957574 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea8a0e4-367b-43d7-a9d3-95887401d9fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:52.960621 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea8a0e4-367b-43d7-a9d3-95887401d9fb admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:53.013055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 659/1326] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:07:52 2022] POST /identity/v3/projects => generated 405 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:53.017389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061095c8-b9d9-4472-97b1-004cfb13089e None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:53.031467 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061095c8-b9d9-4472-97b1-004cfb13089e None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:53.167380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e759565-261e-4492-8678-1371240cf34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:53.168134 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e759565-261e-4492-8678-1371240cf34a admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:53.169491 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e759565-261e-4492-8678-1371240cf34a admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:53.169930 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e759565-261e-4492-8678-1371240cf34a admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:53.175682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e759565-261e-4492-8678-1371240cf34a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:53.178038 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e759565-261e-4492-8678-1371240cf34a admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:53.221355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 668/1327] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:07:53 2022] POST /identity/v3/users => generated 309 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:07:53.225159 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8a0e4-367b-43d7-a9d3-95887401d9fb admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:53.239349 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8a0e4-367b-43d7-a9d3-95887401d9fb admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:53.254900 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba2af8-9575-4465-984a-8e2e31a17724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:53.255655 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba2af8-9575-4465-984a-8e2e31a17724 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:53.257491 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba2af8-9575-4465-984a-8e2e31a17724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:53.257491 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba2af8-9575-4465-984a-8e2e31a17724 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:53.258094 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba2af8-9575-4465-984a-8e2e31a17724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:53.262098 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba2af8-9575-4465-984a-8e2e31a17724 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:53.289739 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 660/1328] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:53.293870 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e759565-261e-4492-8678-1371240cf34a admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:53.307079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e759565-261e-4492-8678-1371240cf34a admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:53.326781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9725626-ea2d-48f6-97ba-8e32f0a203e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:53.328687 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9725626-ea2d-48f6-97ba-8e32f0a203e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:53.328980 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9725626-ea2d-48f6-97ba-8e32f0a203e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:53.329258 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9725626-ea2d-48f6-97ba-8e32f0a203e7 admin admin] PATH_INFO: `/v3/projects/c9ca0da10d164b8b8dfb187933bf846a/users/15396d32a0d84bb5ad6973407f388416/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:53.376804 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9725626-ea2d-48f6-97ba-8e32f0a203e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9ca0da10d164b8b8dfb187933bf846a, user_id=15396d32a0d84bb5ad6973407f388416, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:53.376804 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9725626-ea2d-48f6-97ba-8e32f0a203e7 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:53.392927 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 669/1329] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:07:53 2022] PUT /identity/v3/projects/c9ca0da10d164b8b8dfb187933bf846a/users/15396d32a0d84bb5ad6973407f388416/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:07:53.393982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba2af8-9575-4465-984a-8e2e31a17724 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:53.411835 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba2af8-9575-4465-984a-8e2e31a17724 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:53.438287 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9725626-ea2d-48f6-97ba-8e32f0a203e7 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:53.439911 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d9725626-ea2d-48f6-97ba-8e32f0a203e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:53.493216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:53.518034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3b4b8-d943-4d42-9810-9771ba53ff2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:53.518934 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3b4b8-d943-4d42-9810-9771ba53ff2d admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:53.520701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3b4b8-d943-4d42-9810-9771ba53ff2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:53.521010 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3b4b8-d943-4d42-9810-9771ba53ff2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:53.521863 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3b4b8-d943-4d42-9810-9771ba53ff2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:53.523489 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3b4b8-d943-4d42-9810-9771ba53ff2d admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:53.576831 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 661/1330] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:53.578791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3b4b8-d943-4d42-9810-9771ba53ff2d admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:53.611350 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3b4b8-d943-4d42-9810-9771ba53ff2d admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:53.617651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9725626-ea2d-48f6-97ba-8e32f0a203e7 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:53.627523 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22ba28-bdbb-448b-818f-9b3db4f0dd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:53.628280 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22ba28-bdbb-448b-818f-9b3db4f0dd2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:53.628646 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22ba28-bdbb-448b-818f-9b3db4f0dd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:53.628938 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22ba28-bdbb-448b-818f-9b3db4f0dd2c admin admin] PATH_INFO: `/v3/projects/c9ca0da10d164b8b8dfb187933bf846a/users/15396d32a0d84bb5ad6973407f388416/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:53.630981 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fcdbeb-c32d-4309-941c-5e990f69d87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:53.631652 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcdbeb-c32d-4309-941c-5e990f69d87d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:53.631937 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcdbeb-c32d-4309-941c-5e990f69d87d admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:53.632220 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcdbeb-c32d-4309-941c-5e990f69d87d admin admin] PATH_INFO: `/v3/projects/8cfd193661244940b517de9a812796f2` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:53.635810 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fcdbeb-c32d-4309-941c-5e990f69d87d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cfd193661244940b517de9a812796f2)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:53.635950 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22ba28-bdbb-448b-818f-9b3db4f0dd2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9ca0da10d164b8b8dfb187933bf846a, user_id=15396d32a0d84bb5ad6973407f388416, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:53.638899 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22ba28-bdbb-448b-818f-9b3db4f0dd2c admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:53.639660 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fcdbeb-c32d-4309-941c-5e990f69d87d admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:53.669114 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 662/1331] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:07:53 2022] PUT /identity/v3/projects/c9ca0da10d164b8b8dfb187933bf846a/users/15396d32a0d84bb5ad6973407f388416/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:07:53.771273 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-97fcdbeb-c32d-4309-941c-5e990f69d87d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cfd193661244940b517de9a812796f2', 'request_id': 'req-97fcdbeb-c32d-4309-941c-5e990f69d87d'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:07:53.783157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-97fcdbeb-c32d-4309-941c-5e990f69d87d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cfd193661244940b517de9a812796f2', 'request_id': 'req-97fcdbeb-c32d-4309-941c-5e990f69d87d'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:07:53.796975 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-97fcdbeb-c32d-4309-941c-5e990f69d87d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cfd193661244940b517de9a812796f2', 'request_id': 'req-97fcdbeb-c32d-4309-941c-5e990f69d87d'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:07:53.827265 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-97fcdbeb-c32d-4309-941c-5e990f69d87d admin admin] The token cache is being invalidate because project 8cfd193661244940b517de9a812796f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:07:53.827580 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-97fcdbeb-c32d-4309-941c-5e990f69d87d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:07:53.829504 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-97fcdbeb-c32d-4309-941c-5e990f69d87d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:07:53.832128 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 670/1332] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:07:53 2022] DELETE /identity/v3/projects/8cfd193661244940b517de9a812796f2 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:07:53.836216 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22ba28-bdbb-448b-818f-9b3db4f0dd2c admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:53.838066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-1b22ba28-bdbb-448b-818f-9b3db4f0dd2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:54.005111 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22ba28-bdbb-448b-818f-9b3db4f0dd2c admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:54.164954 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:54.183929 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc5818-cca4-44e1-af4e-9c6000244029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:54.184697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc5818-cca4-44e1-af4e-9c6000244029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:54.184980 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc5818-cca4-44e1-af4e-9c6000244029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:54.185259 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc5818-cca4-44e1-af4e-9c6000244029 admin admin] PATH_INFO: `/v3/users/7ecc4a3304a84bd286a79cf0f4178887` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:54.188957 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc5818-cca4-44e1-af4e-9c6000244029 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ecc4a3304a84bd286a79cf0f4178887)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:54.190961 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc5818-cca4-44e1-af4e-9c6000244029 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:54.286709 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-93cc5818-cca4-44e1-af4e-9c6000244029 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ecc4a3304a84bd286a79cf0f4178887'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:07:54.287578 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-93cc5818-cca4-44e1-af4e-9c6000244029 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ecc4a3304a84bd286a79cf0f4178887'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:07:54.299443 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-93cc5818-cca4-44e1-af4e-9c6000244029 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ecc4a3304a84bd286a79cf0f4178887'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:07:54.317262 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-93cc5818-cca4-44e1-af4e-9c6000244029 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ecc4a3304a84bd286a79cf0f4178887'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:07:54.332868 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-93cc5818-cca4-44e1-af4e-9c6000244029 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ecc4a3304a84bd286a79cf0f4178887'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:07:54.347805 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 663/1333] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:07:53 2022] DELETE /identity/v3/users/7ecc4a3304a84bd286a79cf0f4178887 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:07:55.955701 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fcdbeb-c32d-4309-941c-5e990f69d87d admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:55.970391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fcdbeb-c32d-4309-941c-5e990f69d87d admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:56.096036 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff5427-c590-4479-a6bb-44144d8af6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:56.097019 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff5427-c590-4479-a6bb-44144d8af6c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:56.097617 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff5427-c590-4479-a6bb-44144d8af6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:56.098054 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff5427-c590-4479-a6bb-44144d8af6c3 admin admin] PATH_INFO: `/v3/projects/21c88da0e7524112bee63371333ddd8d` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:56.104419 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ff5427-c590-4479-a6bb-44144d8af6c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21c88da0e7524112bee63371333ddd8d)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:56.108792 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ff5427-c590-4479-a6bb-44144d8af6c3 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:56.229382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-a1ff5427-c590-4479-a6bb-44144d8af6c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21c88da0e7524112bee63371333ddd8d', 'request_id': 'req-a1ff5427-c590-4479-a6bb-44144d8af6c3'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:07:56.232060 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:56.248272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-a1ff5427-c590-4479-a6bb-44144d8af6c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21c88da0e7524112bee63371333ddd8d', 'request_id': 'req-a1ff5427-c590-4479-a6bb-44144d8af6c3'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:07:56.267433 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-a1ff5427-c590-4479-a6bb-44144d8af6c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21c88da0e7524112bee63371333ddd8d', 'request_id': 'req-a1ff5427-c590-4479-a6bb-44144d8af6c3'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:07:56.286150 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-a1ff5427-c590-4479-a6bb-44144d8af6c3 admin admin] The token cache is being invalidate because project 21c88da0e7524112bee63371333ddd8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:07:56.286724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-a1ff5427-c590-4479-a6bb-44144d8af6c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:07:56.288261 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-a1ff5427-c590-4479-a6bb-44144d8af6c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:07:56.300767 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 671/1334] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:07:55 2022] DELETE /identity/v3/projects/21c88da0e7524112bee63371333ddd8d => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:07:56.302407 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e184f-63f9-4145-9709-7980c5daf3ae None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:56.303279 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e184f-63f9-4145-9709-7980c5daf3ae None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:56.303792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e184f-63f9-4145-9709-7980c5daf3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:56.334368 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca977c-df78-46b6-8a42-41149cc122b5 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:56.334831 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca977c-df78-46b6-8a42-41149cc122b5 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:56.335217 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca977c-df78-46b6-8a42-41149cc122b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:56.342407 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-3c8e184f-63f9-4145-9709-7980c5daf3ae None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:56.346742 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-3c8e184f-63f9-4145-9709-7980c5daf3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:56.398557 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-f2ca977c-df78-46b6-8a42-41149cc122b5 None None] MFA Rules not processed for user `15396d32a0d84bb5ad6973407f388416`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:56.400408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f2ca977c-df78-46b6-8a42-41149cc122b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:56.493602 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 664/1335] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:56.505246 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7ff4c-3831-495b-ba28-c7ff5bd403bf None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:56.505800 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7ff4c-3831-495b-ba28-c7ff5bd403bf None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:56.506254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7ff4c-3831-495b-ba28-c7ff5bd403bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:56.545028 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-1ca7ff4c-3831-495b-ba28-c7ff5bd403bf None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:56.546609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-1ca7ff4c-3831-495b-ba28-c7ff5bd403bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:56.569831 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 665/1336] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:56.574075 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca7ff4c-3831-495b-ba28-c7ff5bd403bf None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:56.590590 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca7ff4c-3831-495b-ba28-c7ff5bd403bf None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:56.602735 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 672/1337] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:56 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:56.609277 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41feb4-e1e9-4e98-bbdc-7da5ce9e3f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:56.609954 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41feb4-e1e9-4e98-bbdc-7da5ce9e3f92 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:56.610244 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41feb4-e1e9-4e98-bbdc-7da5ce9e3f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:56.610529 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41feb4-e1e9-4e98-bbdc-7da5ce9e3f92 admin admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:56.611208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41feb4-e1e9-4e98-bbdc-7da5ce9e3f92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:07:56.611517 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41feb4-e1e9-4e98-bbdc-7da5ce9e3f92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:56.613267 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41feb4-e1e9-4e98-bbdc-7da5ce9e3f92 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:56.644437 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 666/1338] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:07:56.649237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca977c-df78-46b6-8a42-41149cc122b5 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:56.665203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca977c-df78-46b6-8a42-41149cc122b5 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:56.676863 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b41fe9-a97b-433c-b3f5-4264c1100e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:56.677537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b41fe9-a97b-433c-b3f5-4264c1100e3b admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:56.677821 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b41fe9-a97b-433c-b3f5-4264c1100e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:56.678098 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b41fe9-a97b-433c-b3f5-4264c1100e3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:56.679630 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b41fe9-a97b-433c-b3f5-4264c1100e3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:56.681561 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b41fe9-a97b-433c-b3f5-4264c1100e3b admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:56.752738 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 673/1339] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:07:56 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:56.753643 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41feb4-e1e9-4e98-bbdc-7da5ce9e3f92 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:56.771921 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41feb4-e1e9-4e98-bbdc-7da5ce9e3f92 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:56.890610 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:56.908321 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d384913-3797-4118-992a-59fdf37c2d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:56.909151 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d384913-3797-4118-992a-59fdf37c2d11 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:56.924903 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d384913-3797-4118-992a-59fdf37c2d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:56.925202 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d384913-3797-4118-992a-59fdf37c2d11 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:56.927057 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d384913-3797-4118-992a-59fdf37c2d11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:56.931599 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d384913-3797-4118-992a-59fdf37c2d11 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:57.012225 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 667/1340] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:07:56 2022] POST /identity/v3/users => generated 291 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:57.017946 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b41fe9-a97b-433c-b3f5-4264c1100e3b admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:57.033402 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b41fe9-a97b-433c-b3f5-4264c1100e3b admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:57.051291 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d150759-d95f-45bd-8c7f-e1284738d0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:57.051922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d150759-d95f-45bd-8c7f-e1284738d0b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:57.052201 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d150759-d95f-45bd-8c7f-e1284738d0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:57.052471 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d150759-d95f-45bd-8c7f-e1284738d0b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:57.053147 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d150759-d95f-45bd-8c7f-e1284738d0b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:57.061326 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d150759-d95f-45bd-8c7f-e1284738d0b1 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:57.092304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 674/1341] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:57.093623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d384913-3797-4118-992a-59fdf37c2d11 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:57.123056 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d384913-3797-4118-992a-59fdf37c2d11 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:57.141306 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b30b7-a57c-4feb-a219-7f3741f194e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:57.141961 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b30b7-a57c-4feb-a219-7f3741f194e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:57.142253 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b30b7-a57c-4feb-a219-7f3741f194e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:57.142534 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b30b7-a57c-4feb-a219-7f3741f194e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:57.143209 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b30b7-a57c-4feb-a219-7f3741f194e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:57.161766 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b30b7-a57c-4feb-a219-7f3741f194e2 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:57.185065 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 668/1342] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:07:57.189472 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d150759-d95f-45bd-8c7f-e1284738d0b1 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:57.203756 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d150759-d95f-45bd-8c7f-e1284738d0b1 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:57.223625 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e40ddd-5989-402f-a8dd-20b96621b23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:57.224305 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e40ddd-5989-402f-a8dd-20b96621b23b admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:57.224617 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e40ddd-5989-402f-a8dd-20b96621b23b admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:57.224898 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e40ddd-5989-402f-a8dd-20b96621b23b admin admin] PATH_INFO: `/v3/projects/973fddfe996f4a4082c8adb2d87c15b7/users/c9c84e7894584e00982f11cff8258b35/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:57.279927 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e40ddd-5989-402f-a8dd-20b96621b23b admin admin] RBAC: Authorizing `identity:create_grant(project_id=973fddfe996f4a4082c8adb2d87c15b7, user_id=c9c84e7894584e00982f11cff8258b35, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:57.289549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e40ddd-5989-402f-a8dd-20b96621b23b admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:57.329401 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 675/1343] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:07:57 2022] PUT /identity/v3/projects/973fddfe996f4a4082c8adb2d87c15b7/users/c9c84e7894584e00982f11cff8258b35/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:07:57.331266 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de3f1a-7fde-42ac-98cf-20912ebadb1f None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:57.331714 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de3f1a-7fde-42ac-98cf-20912ebadb1f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:57.331902 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de3f1a-7fde-42ac-98cf-20912ebadb1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:57.413376 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-c7de3f1a-7fde-42ac-98cf-20912ebadb1f None None] MFA Rules not processed for user `c9c84e7894584e00982f11cff8258b35`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:57.415031 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c7de3f1a-7fde-42ac-98cf-20912ebadb1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:57.604050 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 669/1344] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:57.606016 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e40ddd-5989-402f-a8dd-20b96621b23b admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:57.753797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e40ddd-5989-402f-a8dd-20b96621b23b admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:57.772081 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da4749-844f-4d8f-96a9-5f35b832344c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:57.772751 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da4749-844f-4d8f-96a9-5f35b832344c admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:57.773031 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da4749-844f-4d8f-96a9-5f35b832344c admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:57.773332 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da4749-844f-4d8f-96a9-5f35b832344c admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:57.774660 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4da4749-844f-4d8f-96a9-5f35b832344c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:57.776552 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4da4749-844f-4d8f-96a9-5f35b832344c admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:57.874986 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:57.920040 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 676/1345] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:07:57 2022] POST /identity/v3/projects => generated 379 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:57.924840 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de3f1a-7fde-42ac-98cf-20912ebadb1f None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:57.941535 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de3f1a-7fde-42ac-98cf-20912ebadb1f None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:58.117699 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab4dc54-380c-4b71-9644-8a7836ec666c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:58.118342 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4dc54-380c-4b71-9644-8a7836ec666c admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:58.118627 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4dc54-380c-4b71-9644-8a7836ec666c admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:58.118920 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4dc54-380c-4b71-9644-8a7836ec666c admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:58.119709 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab4dc54-380c-4b71-9644-8a7836ec666c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:58.138682 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab4dc54-380c-4b71-9644-8a7836ec666c admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:58.216815 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da4749-844f-4d8f-96a9-5f35b832344c admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:58.218727 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 670/1346] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:07:57 2022] POST /identity/v3/users => generated 296 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:07:58.245707 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da4749-844f-4d8f-96a9-5f35b832344c admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:58.259875 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e8dd5-e80b-4e4a-a510-494e5de8c28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:58.260636 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e8dd5-e80b-4e4a-a510-494e5de8c28f admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:58.263534 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e8dd5-e80b-4e4a-a510-494e5de8c28f admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:58.263846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e8dd5-e80b-4e4a-a510-494e5de8c28f admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:58.264681 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e8dd5-e80b-4e4a-a510-494e5de8c28f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:58.268685 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e8dd5-e80b-4e4a-a510-494e5de8c28f admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:58.299806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab4dc54-380c-4b71-9644-8a7836ec666c admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:58.307047 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 677/1347] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:58 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:58.318213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab4dc54-380c-4b71-9644-8a7836ec666c admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:58.324073 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901d0da-db95-42b1-a9e4-edf8215b34c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:58.325035 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901d0da-db95-42b1-a9e4-edf8215b34c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:58.325328 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901d0da-db95-42b1-a9e4-edf8215b34c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:58.325612 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901d0da-db95-42b1-a9e4-edf8215b34c7 admin admin] PATH_INFO: `/v3/projects/0ad4c46378764a42b925976d51254ab7/users/e2aefaf1ff3a4d1da56b6912d3642104/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:58.366703 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901d0da-db95-42b1-a9e4-edf8215b34c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ad4c46378764a42b925976d51254ab7, user_id=e2aefaf1ff3a4d1da56b6912d3642104, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:58.375439 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901d0da-db95-42b1-a9e4-edf8215b34c7 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:58.401897 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 671/1348] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:07:58 2022] PUT /identity/v3/projects/0ad4c46378764a42b925976d51254ab7/users/e2aefaf1ff3a4d1da56b6912d3642104/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:07:58.405600 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e8dd5-e80b-4e4a-a510-494e5de8c28f admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:58.424535 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e8dd5-e80b-4e4a-a510-494e5de8c28f admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:58.521254 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d652fd-8c3a-4dd1-9456-606714c033f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:58.522661 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d652fd-8c3a-4dd1-9456-606714c033f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:58.522962 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d652fd-8c3a-4dd1-9456-606714c033f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:58.523249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d652fd-8c3a-4dd1-9456-606714c033f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:58.523956 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d652fd-8c3a-4dd1-9456-606714c033f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:58.526360 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d652fd-8c3a-4dd1-9456-606714c033f3 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:58.549403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 678/1349] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:07:58 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:07:58.551566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901d0da-db95-42b1-a9e4-edf8215b34c7 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:58.566356 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901d0da-db95-42b1-a9e4-edf8215b34c7 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:58.582137 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa09d4-8481-4d10-8fed-beb60872cf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:58.583448 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa09d4-8481-4d10-8fed-beb60872cf20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:58.584015 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa09d4-8481-4d10-8fed-beb60872cf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:58.584745 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa09d4-8481-4d10-8fed-beb60872cf20 admin admin] PATH_INFO: `/v3/projects/0ad4c46378764a42b925976d51254ab7/users/e2aefaf1ff3a4d1da56b6912d3642104/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:58.587697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa09d4-8481-4d10-8fed-beb60872cf20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ad4c46378764a42b925976d51254ab7, user_id=e2aefaf1ff3a4d1da56b6912d3642104, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:58.590742 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa09d4-8481-4d10-8fed-beb60872cf20 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:58.634915 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb52923-499b-4358-aaf4-b00f73eacb82 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:58.640502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb52923-499b-4358-aaf4-b00f73eacb82 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:58.640970 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb52923-499b-4358-aaf4-b00f73eacb82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:58.646087 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 672/1350] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:07:58 2022] PUT /identity/v3/projects/0ad4c46378764a42b925976d51254ab7/users/e2aefaf1ff3a4d1da56b6912d3642104/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:07:58.692014 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-9fb52923-499b-4358-aaf4-b00f73eacb82 None None] MFA Rules not processed for user `e2aefaf1ff3a4d1da56b6912d3642104`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:58.696263 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9fb52923-499b-4358-aaf4-b00f73eacb82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:58.860412 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa09d4-8481-4d10-8fed-beb60872cf20 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:58.861771 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 679/1351] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:07:58 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:58.862573 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-edaa09d4-8481-4d10-8fed-beb60872cf20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:58.931132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:07:59.071615 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa09d4-8481-4d10-8fed-beb60872cf20 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:59.085612 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac328525-5019-47c1-b3c6-162998761306 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:59.086257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac328525-5019-47c1-b3c6-162998761306 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:59.086546 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac328525-5019-47c1-b3c6-162998761306 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:59.086806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac328525-5019-47c1-b3c6-162998761306 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:59.131957 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac328525-5019-47c1-b3c6-162998761306 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:59.133464 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac328525-5019-47c1-b3c6-162998761306 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:59.163883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 673/1352] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:07:58 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:07:59.276068 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5dfd9-ee08-42e7-9a54-574f23de164a None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:59.276648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5dfd9-ee08-42e7-9a54-574f23de164a None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:59.276778 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5dfd9-ee08-42e7-9a54-574f23de164a None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:59.278293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 680/1353] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 00:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:07:59.337942 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcba2e-2e29-480e-8957-b93760c6e9ce None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:59.338570 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcba2e-2e29-480e-8957-b93760c6e9ce None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:59.339001 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcba2e-2e29-480e-8957-b93760c6e9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:59.447559 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-7ffcba2e-2e29-480e-8957-b93760c6e9ce None None] MFA Rules not processed for user `24108b6355414f64ba5e7bf84011eee6`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:07:59.448821 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-7ffcba2e-2e29-480e-8957-b93760c6e9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:59.677340 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 674/1354] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 00:07:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:07:59.706962 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5dfd9-ee08-42e7-9a54-574f23de164a None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:07:59.709146 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-40b5dfd9-ee08-42e7-9a54-574f23de164a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:07:59.742339 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5dfd9-ee08-42e7-9a54-574f23de164a None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:07:59.754399 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f6822-b30e-4be4-9c28-063533569c2f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24108b6355414f64ba5e7bf84011eee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:07:59.755030 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f6822-b30e-4be4-9c28-063533569c2f service placement] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:07:59.755315 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f6822-b30e-4be4-9c28-063533569c2f service placement] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:07:59.755596 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f6822-b30e-4be4-9c28-063533569c2f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:07:59.772629 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f6822-b30e-4be4-9c28-063533569c2f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:07:59.775948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f6822-b30e-4be4-9c28-063533569c2f service placement] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:07:59.801520 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 681/1355] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:07:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:08:00.032691 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcba2e-2e29-480e-8957-b93760c6e9ce None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:00.056325 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcba2e-2e29-480e-8957-b93760c6e9ce None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:00.072622 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8814184-6a4c-4e6f-9ac7-3ca4385c5f7f tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2aefaf1ff3a4d1da56b6912d3642104', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad4c46378764a42b925976d51254ab7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:00.073264 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8814184-6a4c-4e6f-9ac7-3ca4385c5f7f tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:00.073555 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8814184-6a4c-4e6f-9ac7-3ca4385c5f7f tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:00.073837 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8814184-6a4c-4e6f-9ac7-3ca4385c5f7f tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:00.074515 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8814184-6a4c-4e6f-9ac7-3ca4385c5f7f tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:08:00.074827 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8814184-6a4c-4e6f-9ac7-3ca4385c5f7f tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:00.076450 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8814184-6a4c-4e6f-9ac7-3ca4385c5f7f tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:00.121558 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 675/1356] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:08:00.125877 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f6822-b30e-4be4-9c28-063533569c2f service placement] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:00.141147 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f6822-b30e-4be4-9c28-063533569c2f service placement] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:00.153822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6afbd6-2adb-4533-ab1f-51aae0982099 tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2aefaf1ff3a4d1da56b6912d3642104', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad4c46378764a42b925976d51254ab7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:00.154714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6afbd6-2adb-4533-ab1f-51aae0982099 tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:00.155253 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6afbd6-2adb-4533-ab1f-51aae0982099 tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:00.155749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6afbd6-2adb-4533-ab1f-51aae0982099 tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:00.156796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6afbd6-2adb-4533-ab1f-51aae0982099 tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:00.159573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6afbd6-2adb-4533-ab1f-51aae0982099 tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:00.284185 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8814184-6a4c-4e6f-9ac7-3ca4385c5f7f tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:00.284542 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 682/1357] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:00 2022] POST /identity/v3/projects => generated 353 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:00.299081 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8814184-6a4c-4e6f-9ac7-3ca4385c5f7f tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:00.421239 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35acec88-eb01-4d15-9970-9e716d95b99b tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2aefaf1ff3a4d1da56b6912d3642104', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad4c46378764a42b925976d51254ab7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:00.422270 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35acec88-eb01-4d15-9970-9e716d95b99b tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:00.423402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35acec88-eb01-4d15-9970-9e716d95b99b tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:00.423985 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35acec88-eb01-4d15-9970-9e716d95b99b tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:00.425032 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35acec88-eb01-4d15-9970-9e716d95b99b tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:08:00.425576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35acec88-eb01-4d15-9970-9e716d95b99b tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:00.428145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35acec88-eb01-4d15-9970-9e716d95b99b tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:00.447482 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 676/1358] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:08:00.529798 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6afbd6-2adb-4533-ab1f-51aae0982099 tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:00.543231 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6afbd6-2adb-4533-ab1f-51aae0982099 tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:00.558840 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d59be2-a43e-4908-8716-e2997e90c8e7 tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e2aefaf1ff3a4d1da56b6912d3642104', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad4c46378764a42b925976d51254ab7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:00.559888 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d59be2-a43e-4908-8716-e2997e90c8e7 tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:00.560337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d59be2-a43e-4908-8716-e2997e90c8e7 tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:00.560832 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d59be2-a43e-4908-8716-e2997e90c8e7 tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] PATH_INFO: `/v3/projects/b08d54de23644e9192bf0b79fd9e37ad` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:00.562683 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d59be2-a43e-4908-8716-e2997e90c8e7 tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b08d54de23644e9192bf0b79fd9e37ad)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:00.565084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d59be2-a43e-4908-8716-e2997e90c8e7 tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:00.651970 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:INSERT=2,keystone:SELECT=97,keystone:DELETE=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:00.724078 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-90d59be2-a43e-4908-8716-e2997e90c8e7 tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b08d54de23644e9192bf0b79fd9e37ad', 'request_id': 'req-90d59be2-a43e-4908-8716-e2997e90c8e7'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:00.741629 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-90d59be2-a43e-4908-8716-e2997e90c8e7 tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b08d54de23644e9192bf0b79fd9e37ad', 'request_id': 'req-90d59be2-a43e-4908-8716-e2997e90c8e7'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:00.755828 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-90d59be2-a43e-4908-8716-e2997e90c8e7 tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b08d54de23644e9192bf0b79fd9e37ad', 'request_id': 'req-90d59be2-a43e-4908-8716-e2997e90c8e7'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:00.774353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-90d59be2-a43e-4908-8716-e2997e90c8e7 tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] The token cache is being invalidate because project b08d54de23644e9192bf0b79fd9e37ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:00.774848 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-90d59be2-a43e-4908-8716-e2997e90c8e7 tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:00.775707 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-90d59be2-a43e-4908-8716-e2997e90c8e7 tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:00.778461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 683/1359] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:00 2022] DELETE /identity/v3/projects/b08d54de23644e9192bf0b79fd9e37ad => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:00.960337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35acec88-eb01-4d15-9970-9e716d95b99b tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:00.962629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-35acec88-eb01-4d15-9970-9e716d95b99b tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:01.114492 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:01.209223 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35acec88-eb01-4d15-9970-9e716d95b99b tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:01.397875 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0f959-52f0-4f48-b4c8-d44db9233cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:01.398668 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0f959-52f0-4f48-b4c8-d44db9233cbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:01.399058 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0f959-52f0-4f48-b4c8-d44db9233cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:01.399450 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0f959-52f0-4f48-b4c8-d44db9233cbb admin admin] PATH_INFO: `/v3/users/a3ea1c4cb0894cc7a6202caeffa38bc9` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:01.408985 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c0f959-52f0-4f48-b4c8-d44db9233cbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3ea1c4cb0894cc7a6202caeffa38bc9)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:01.411163 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c0f959-52f0-4f48-b4c8-d44db9233cbb admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:01.422077 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d59be2-a43e-4908-8716-e2997e90c8e7 tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:01.424690 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-90d59be2-a43e-4908-8716-e2997e90c8e7 tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:01.465510 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d59be2-a43e-4908-8716-e2997e90c8e7 tempest-VolumeQuotaClassesTest-846031264 tempest-VolumeQuotaClassesTest-846031264-project-admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:01.487411 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94307a-5aa1-4a5c-ab6c-afc259ba00e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:01.488790 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94307a-5aa1-4a5c-ab6c-afc259ba00e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:01.496976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94307a-5aa1-4a5c-ab6c-afc259ba00e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:01.497445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94307a-5aa1-4a5c-ab6c-afc259ba00e9 admin admin] PATH_INFO: `/v3/users/c9c84e7894584e00982f11cff8258b35` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:01.503273 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b94307a-5aa1-4a5c-ab6c-afc259ba00e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9c84e7894584e00982f11cff8258b35)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:01.507789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: WARNING py.warnings [None req-1b94307a-5aa1-4a5c-ab6c-afc259ba00e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:08:01.507789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: warnings.warn(msg) Oct 23 00:08:01.507789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: Oct 23 00:08:01.510721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b94307a-5aa1-4a5c-ab6c-afc259ba00e9 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:01.591861 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-80c0f959-52f0-4f48-b4c8-d44db9233cbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3ea1c4cb0894cc7a6202caeffa38bc9'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:01.594664 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-80c0f959-52f0-4f48-b4c8-d44db9233cbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3ea1c4cb0894cc7a6202caeffa38bc9'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:01.621478 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-80c0f959-52f0-4f48-b4c8-d44db9233cbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3ea1c4cb0894cc7a6202caeffa38bc9'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:01.661041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-80c0f959-52f0-4f48-b4c8-d44db9233cbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3ea1c4cb0894cc7a6202caeffa38bc9'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:01.682439 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-80c0f959-52f0-4f48-b4c8-d44db9233cbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3ea1c4cb0894cc7a6202caeffa38bc9'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:01.727123 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 677/1360] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:00 2022] DELETE /identity/v3/users/a3ea1c4cb0894cc7a6202caeffa38bc9 => generated 0 bytes in 763 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:01.743478 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1b94307a-5aa1-4a5c-ab6c-afc259ba00e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9c84e7894584e00982f11cff8258b35'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:01.783994 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1b94307a-5aa1-4a5c-ab6c-afc259ba00e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9c84e7894584e00982f11cff8258b35'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:01.851997 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1b94307a-5aa1-4a5c-ab6c-afc259ba00e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9c84e7894584e00982f11cff8258b35'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:01.853063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1b94307a-5aa1-4a5c-ab6c-afc259ba00e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9c84e7894584e00982f11cff8258b35'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:01.854028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1b94307a-5aa1-4a5c-ab6c-afc259ba00e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9c84e7894584e00982f11cff8258b35'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:01.878258 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 684/1361] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:01 2022] DELETE /identity/v3/users/c9c84e7894584e00982f11cff8258b35 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:01.882746 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94810bb8-7b65-40e8-bc5c-03e8ea8453b4 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:01.883041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94810bb8-7b65-40e8-bc5c-03e8ea8453b4 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:01.883324 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94810bb8-7b65-40e8-bc5c-03e8ea8453b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:01.919723 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-94810bb8-7b65-40e8-bc5c-03e8ea8453b4 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:01.921754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-94810bb8-7b65-40e8-bc5c-03e8ea8453b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:02.080147 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 678/1362] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:02.120234 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94307a-5aa1-4a5c-ab6c-afc259ba00e9 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:02.121754 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1b94307a-5aa1-4a5c-ab6c-afc259ba00e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:02.247543 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807155df-8f2e-4734-ba5b-9bd7beb3d2ee None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:02.247847 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807155df-8f2e-4734-ba5b-9bd7beb3d2ee None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:02.248134 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807155df-8f2e-4734-ba5b-9bd7beb3d2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:02.258911 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:02.286592 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94307a-5aa1-4a5c-ab6c-afc259ba00e9 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:02.295632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ea59a-1eb7-4b4b-803a-8ccb44c2b38e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:02.296842 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ea59a-1eb7-4b4b-803a-8ccb44c2b38e service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:02.297129 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ea59a-1eb7-4b4b-803a-8ccb44c2b38e service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:02.298184 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ea59a-1eb7-4b4b-803a-8ccb44c2b38e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:02.317048 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-807155df-8f2e-4734-ba5b-9bd7beb3d2ee None None] MFA Rules not processed for user `8b4ef165353f4f1189400c005519c2ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:02.318296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-807155df-8f2e-4734-ba5b-9bd7beb3d2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:02.325077 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ea59a-1eb7-4b4b-803a-8ccb44c2b38e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:02.328443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ea59a-1eb7-4b4b-803a-8ccb44c2b38e service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:02.356954 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 685/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:02.461015 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 679/1364] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:02.478513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ea59a-1eb7-4b4b-803a-8ccb44c2b38e service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:02.482053 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-310ea59a-1eb7-4b4b-803a-8ccb44c2b38e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:02.643828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807155df-8f2e-4734-ba5b-9bd7beb3d2ee None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:02.646248 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-807155df-8f2e-4734-ba5b-9bd7beb3d2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:02.674142 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807155df-8f2e-4734-ba5b-9bd7beb3d2ee None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:02.687304 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c25699-d231-4fbb-85d7-b498729f0d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:02.688043 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c25699-d231-4fbb-85d7-b498729f0d7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:02.688402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c25699-d231-4fbb-85d7-b498729f0d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:02.688740 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c25699-d231-4fbb-85d7-b498729f0d7e admin admin] PATH_INFO: `/v3/projects/7cd8dc6fdb244b24bdaf951caabc10e2` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:02.690423 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c25699-d231-4fbb-85d7-b498729f0d7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cd8dc6fdb244b24bdaf951caabc10e2)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:02.691563 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: WARNING py.warnings [None req-34c25699-d231-4fbb-85d7-b498729f0d7e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:08:02.691563 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: warnings.warn(msg) Oct 23 00:08:02.691563 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: Oct 23 00:08:02.692910 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c25699-d231-4fbb-85d7-b498729f0d7e admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:02.775275 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:02.817025 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-34c25699-d231-4fbb-85d7-b498729f0d7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cd8dc6fdb244b24bdaf951caabc10e2', 'request_id': 'req-34c25699-d231-4fbb-85d7-b498729f0d7e'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:02.837134 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-34c25699-d231-4fbb-85d7-b498729f0d7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cd8dc6fdb244b24bdaf951caabc10e2', 'request_id': 'req-34c25699-d231-4fbb-85d7-b498729f0d7e'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:02.862170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-34c25699-d231-4fbb-85d7-b498729f0d7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cd8dc6fdb244b24bdaf951caabc10e2', 'request_id': 'req-34c25699-d231-4fbb-85d7-b498729f0d7e'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:02.884322 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ea59a-1eb7-4b4b-803a-8ccb44c2b38e service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:02.896776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-34c25699-d231-4fbb-85d7-b498729f0d7e admin admin] The token cache is being invalidate because project 7cd8dc6fdb244b24bdaf951caabc10e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:02.898091 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-34c25699-d231-4fbb-85d7-b498729f0d7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:02.900124 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-34c25699-d231-4fbb-85d7-b498729f0d7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:02.911859 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 680/1365] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:02 2022] DELETE /identity/v3/projects/7cd8dc6fdb244b24bdaf951caabc10e2 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:02.934479 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c25699-d231-4fbb-85d7-b498729f0d7e admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:02.936997 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-34c25699-d231-4fbb-85d7-b498729f0d7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:03.036869 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e8339-19c7-4cee-80fc-3eb39ee15fc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:03.037672 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e8339-19c7-4cee-80fc-3eb39ee15fc4 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:03.038073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e8339-19c7-4cee-80fc-3eb39ee15fc4 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:03.038461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e8339-19c7-4cee-80fc-3eb39ee15fc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:03.042544 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-fc4e8339-19c7-4cee-80fc-3eb39ee15fc4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:03.223483 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c25699-d231-4fbb-85d7-b498729f0d7e admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:03.248053 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee4fda-57a5-4b38-afd2-3e8f7cfa9fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:03.249383 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee4fda-57a5-4b38-afd2-3e8f7cfa9fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:03.250481 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee4fda-57a5-4b38-afd2-3e8f7cfa9fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:03.250901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee4fda-57a5-4b38-afd2-3e8f7cfa9fda admin admin] PATH_INFO: `/v3/users/15396d32a0d84bb5ad6973407f388416` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:03.253881 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee4fda-57a5-4b38-afd2-3e8f7cfa9fda admin admin] RBAC: Authorizing `identity:delete_user(user_id=15396d32a0d84bb5ad6973407f388416)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:03.256384 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4e8339-19c7-4cee-80fc-3eb39ee15fc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:03.258105 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4e8339-19c7-4cee-80fc-3eb39ee15fc4 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:03.258746 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ee4fda-57a5-4b38-afd2-3e8f7cfa9fda admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:03.296725 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 686/1366] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 818 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:03.304974 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:03.382677 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-16ee4fda-57a5-4b38-afd2-3e8f7cfa9fda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15396d32a0d84bb5ad6973407f388416'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:03.385755 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-16ee4fda-57a5-4b38-afd2-3e8f7cfa9fda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15396d32a0d84bb5ad6973407f388416'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:03.421927 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-16ee4fda-57a5-4b38-afd2-3e8f7cfa9fda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15396d32a0d84bb5ad6973407f388416'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:03.474255 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-16ee4fda-57a5-4b38-afd2-3e8f7cfa9fda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15396d32a0d84bb5ad6973407f388416'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:03.494801 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-16ee4fda-57a5-4b38-afd2-3e8f7cfa9fda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15396d32a0d84bb5ad6973407f388416'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:03.505817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 681/1367] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:02 2022] DELETE /identity/v3/users/15396d32a0d84bb5ad6973407f388416 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:03.722744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8ee29-9749-497f-8fec-c46c422a5c8a None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:03.723620 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8ee29-9749-497f-8fec-c46c422a5c8a None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:03.724983 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8ee29-9749-497f-8fec-c46c422a5c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:03.799734 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-e3a8ee29-9749-497f-8fec-c46c422a5c8a None None] MFA Rules not processed for user `127eff42dfb141c3b78ce04636af1f28`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:03.801642 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e3a8ee29-9749-497f-8fec-c46c422a5c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:03.814035 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee4fda-57a5-4b38-afd2-3e8f7cfa9fda admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:03.815518 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-16ee4fda-57a5-4b38-afd2-3e8f7cfa9fda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:03.897309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:03.931712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 687/1368] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:03.933293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a8ee29-9749-497f-8fec-c46c422a5c8a None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:03.943253 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e3a8ee29-9749-497f-8fec-c46c422a5c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:03.945832 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ee4fda-57a5-4b38-afd2-3e8f7cfa9fda admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:03.963477 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a8ee29-9749-497f-8fec-c46c422a5c8a None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:03.971614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f546bdc-3601-4653-806c-f30f7bc23957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:03.972367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f546bdc-3601-4653-806c-f30f7bc23957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:03.972763 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f546bdc-3601-4653-806c-f30f7bc23957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:03.973172 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f546bdc-3601-4653-806c-f30f7bc23957 admin admin] PATH_INFO: `/v3/projects/c9ca0da10d164b8b8dfb187933bf846a` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:03.974564 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f546bdc-3601-4653-806c-f30f7bc23957 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9ca0da10d164b8b8dfb187933bf846a)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:03.977298 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f546bdc-3601-4653-806c-f30f7bc23957 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:03.977497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd31b1-84e7-4c76-87b4-cb8d9703acfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:03.978454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd31b1-84e7-4c76-87b4-cb8d9703acfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:03.979145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd31b1-84e7-4c76-87b4-cb8d9703acfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:03.979646 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd31b1-84e7-4c76-87b4-cb8d9703acfd admin admin] PATH_INFO: `/v3/projects/973fddfe996f4a4082c8adb2d87c15b7` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:03.981626 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd31b1-84e7-4c76-87b4-cb8d9703acfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=973fddfe996f4a4082c8adb2d87c15b7)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:03.984352 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd31b1-84e7-4c76-87b4-cb8d9703acfd admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:04.074261 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-50dd31b1-84e7-4c76-87b4-cb8d9703acfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '973fddfe996f4a4082c8adb2d87c15b7', 'request_id': 'req-50dd31b1-84e7-4c76-87b4-cb8d9703acfd'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:04.089666 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-50dd31b1-84e7-4c76-87b4-cb8d9703acfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '973fddfe996f4a4082c8adb2d87c15b7', 'request_id': 'req-50dd31b1-84e7-4c76-87b4-cb8d9703acfd'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:04.090631 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-7f546bdc-3601-4653-806c-f30f7bc23957 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9ca0da10d164b8b8dfb187933bf846a', 'request_id': 'req-7f546bdc-3601-4653-806c-f30f7bc23957'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:04.107818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-50dd31b1-84e7-4c76-87b4-cb8d9703acfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '973fddfe996f4a4082c8adb2d87c15b7', 'request_id': 'req-50dd31b1-84e7-4c76-87b4-cb8d9703acfd'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:04.108718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-7f546bdc-3601-4653-806c-f30f7bc23957 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9ca0da10d164b8b8dfb187933bf846a', 'request_id': 'req-7f546bdc-3601-4653-806c-f30f7bc23957'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:04.121991 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-7f546bdc-3601-4653-806c-f30f7bc23957 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9ca0da10d164b8b8dfb187933bf846a', 'request_id': 'req-7f546bdc-3601-4653-806c-f30f7bc23957'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:04.127967 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-50dd31b1-84e7-4c76-87b4-cb8d9703acfd admin admin] The token cache is being invalidate because project 973fddfe996f4a4082c8adb2d87c15b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:04.128580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-50dd31b1-84e7-4c76-87b4-cb8d9703acfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:04.129346 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-50dd31b1-84e7-4c76-87b4-cb8d9703acfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:04.132175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 682/1369] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:03 2022] DELETE /identity/v3/projects/973fddfe996f4a4082c8adb2d87c15b7 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:08:04.134663 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10686a41-327e-41af-9020-f7fd1a8a60f9 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:04.135186 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10686a41-327e-41af-9020-f7fd1a8a60f9 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:04.135540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10686a41-327e-41af-9020-f7fd1a8a60f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:04.156450 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-7f546bdc-3601-4653-806c-f30f7bc23957 admin admin] The token cache is being invalidate because project c9ca0da10d164b8b8dfb187933bf846a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:04.163067 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-7f546bdc-3601-4653-806c-f30f7bc23957 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:04.163832 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-7f546bdc-3601-4653-806c-f30f7bc23957 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:04.165655 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 688/1370] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:03 2022] DELETE /identity/v3/projects/c9ca0da10d164b8b8dfb187933bf846a => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:04.166666 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556c84c-8705-40f6-a78c-1ea454a3aab5 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:04.170658 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556c84c-8705-40f6-a78c-1ea454a3aab5 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:04.171078 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556c84c-8705-40f6-a78c-1ea454a3aab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:04.174810 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-10686a41-327e-41af-9020-f7fd1a8a60f9 None None] MFA Rules not processed for user `f95f4fc0d5c5428c96545f543139489b`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:04.176215 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-10686a41-327e-41af-9020-f7fd1a8a60f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:04.192961 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:04.208675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-e556c84c-8705-40f6-a78c-1ea454a3aab5 None None] MFA Rules not processed for user `127eff42dfb141c3b78ce04636af1f28`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:04.210675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e556c84c-8705-40f6-a78c-1ea454a3aab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:04.304304 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 683/1371] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:04.305579 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10686a41-327e-41af-9020-f7fd1a8a60f9 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:04.315832 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-10686a41-327e-41af-9020-f7fd1a8a60f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:04.356068 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 689/1372] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:04.357611 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b101878-083c-4e2f-a287-d630b9f6dd80 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:04.358063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b101878-083c-4e2f-a287-d630b9f6dd80 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:04.359073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b101878-083c-4e2f-a287-d630b9f6dd80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:04.385993 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1b101878-083c-4e2f-a287-d630b9f6dd80 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:04.387688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1b101878-083c-4e2f-a287-d630b9f6dd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:04.479353 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10686a41-327e-41af-9020-f7fd1a8a60f9 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:04.490111 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 690/1373] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:04.493766 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe3e18-fdf6-48ac-9e06-8e9332b212e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:04.494083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1df76-39ce-4128-a3d0-484f21365fc1 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:04.494483 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1df76-39ce-4128-a3d0-484f21365fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:04.494874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1df76-39ce-4128-a3d0-484f21365fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:04.495558 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe3e18-fdf6-48ac-9e06-8e9332b212e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:04.495997 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe3e18-fdf6-48ac-9e06-8e9332b212e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:04.496470 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe3e18-fdf6-48ac-9e06-8e9332b212e0 admin admin] PATH_INFO: `/v3/users/e2aefaf1ff3a4d1da56b6912d3642104` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:04.497909 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fe3e18-fdf6-48ac-9e06-8e9332b212e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2aefaf1ff3a4d1da56b6912d3642104)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:04.503396 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fe3e18-fdf6-48ac-9e06-8e9332b212e0 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:04.527381 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-02a1df76-39ce-4128-a3d0-484f21365fc1 None None] MFA Rules not processed for user `f95f4fc0d5c5428c96545f543139489b`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:04.530745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-02a1df76-39ce-4128-a3d0-484f21365fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:04.538505 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 691/1374] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:04.543867 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1df76-39ce-4128-a3d0-484f21365fc1 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:04.545459 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-02a1df76-39ce-4128-a3d0-484f21365fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:04.573285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:04.609678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-80fe3e18-fdf6-48ac-9e06-8e9332b212e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2aefaf1ff3a4d1da56b6912d3642104'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:04.612292 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-80fe3e18-fdf6-48ac-9e06-8e9332b212e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2aefaf1ff3a4d1da56b6912d3642104'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:04.628191 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-80fe3e18-fdf6-48ac-9e06-8e9332b212e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2aefaf1ff3a4d1da56b6912d3642104'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:04.653115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1df76-39ce-4128-a3d0-484f21365fc1 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:04.659076 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-80fe3e18-fdf6-48ac-9e06-8e9332b212e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2aefaf1ff3a4d1da56b6912d3642104'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:04.660101 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb9474-d750-4da1-90b8-e6f645eb688e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:04.661014 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb9474-d750-4da1-90b8-e6f645eb688e service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:04.661355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb9474-d750-4da1-90b8-e6f645eb688e service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:04.661836 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb9474-d750-4da1-90b8-e6f645eb688e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:04.663939 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-6fbb9474-d750-4da1-90b8-e6f645eb688e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:04.669226 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-80fe3e18-fdf6-48ac-9e06-8e9332b212e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2aefaf1ff3a4d1da56b6912d3642104'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:04.678766 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb9474-d750-4da1-90b8-e6f645eb688e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:04.680774 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb9474-d750-4da1-90b8-e6f645eb688e service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:04.681521 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 684/1375] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:04 2022] DELETE /identity/v3/users/e2aefaf1ff3a4d1da56b6912d3642104 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:04.683416 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038fcd3-2635-4dd8-b43f-9aef92d92467 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:04.683811 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038fcd3-2635-4dd8-b43f-9aef92d92467 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:04.694743 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 692/1376] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:04.696303 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb9474-d750-4da1-90b8-e6f645eb688e service nova] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:04.697793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-6fbb9474-d750-4da1-90b8-e6f645eb688e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:04.698555 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038fcd3-2635-4dd8-b43f-9aef92d92467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:04.718782 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb9474-d750-4da1-90b8-e6f645eb688e service nova] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:04.762926 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-0038fcd3-2635-4dd8-b43f-9aef92d92467 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:04.765617 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-0038fcd3-2635-4dd8-b43f-9aef92d92467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:04.833567 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679719f7-2a1c-4809-bfcd-f5a78e65fae6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:04.858044 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679719f7-2a1c-4809-bfcd-f5a78e65fae6 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:04.858633 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679719f7-2a1c-4809-bfcd-f5a78e65fae6 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:04.858789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:04.859334 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679719f7-2a1c-4809-bfcd-f5a78e65fae6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:04.865625 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-679719f7-2a1c-4809-bfcd-f5a78e65fae6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:04.979375 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679719f7-2a1c-4809-bfcd-f5a78e65fae6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:04.981292 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679719f7-2a1c-4809-bfcd-f5a78e65fae6 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:04.991736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 693/1377] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:04 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:05.239216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679719f7-2a1c-4809-bfcd-f5a78e65fae6 service nova] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:05.251078 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 685/1378] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 569 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:05.252694 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679719f7-2a1c-4809-bfcd-f5a78e65fae6 service nova] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:05.258399 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd4d4f-adfa-4d79-948b-ec964af66f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:05.259178 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd4d4f-adfa-4d79-948b-ec964af66f84 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:05.259625 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd4d4f-adfa-4d79-948b-ec964af66f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:05.260086 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd4d4f-adfa-4d79-948b-ec964af66f84 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:05.260788 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dd4d4f-adfa-4d79-948b-ec964af66f84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:08:05.261101 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dd4d4f-adfa-4d79-948b-ec964af66f84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:05.262763 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dd4d4f-adfa-4d79-948b-ec964af66f84 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:05.284555 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038fcd3-2635-4dd8-b43f-9aef92d92467 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:05.291477 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 694/1379] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:08:05.298506 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038fcd3-2635-4dd8-b43f-9aef92d92467 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:05.306990 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b2887-ce9d-42e9-a64f-744efa2ef3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:05.307690 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b2887-ce9d-42e9-a64f-744efa2ef3a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:05.307690 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b2887-ce9d-42e9-a64f-744efa2ef3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:05.308159 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836b2887-ce9d-42e9-a64f-744efa2ef3a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:05.309147 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836b2887-ce9d-42e9-a64f-744efa2ef3a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:05.311243 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836b2887-ce9d-42e9-a64f-744efa2ef3a6 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:05.385286 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd4d4f-adfa-4d79-948b-ec964af66f84 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:05.396226 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 686/1380] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:05 2022] POST /identity/v3/projects => generated 365 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:05.401567 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd4d4f-adfa-4d79-948b-ec964af66f84 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:05.465864 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:05.580073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3386c-9b63-40c0-be30-e7a2d026f97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:05.580839 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3386c-9b63-40c0-be30-e7a2d026f97e admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:05.584789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3386c-9b63-40c0-be30-e7a2d026f97e admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:05.585123 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3386c-9b63-40c0-be30-e7a2d026f97e admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:05.588919 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac3386c-9b63-40c0-be30-e7a2d026f97e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:05.602541 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac3386c-9b63-40c0-be30-e7a2d026f97e admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:05.663286 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b2887-ce9d-42e9-a64f-744efa2ef3a6 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:05.664438 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 695/1381] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:05 2022] POST /identity/v3/users => generated 283 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:05.682012 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836b2887-ce9d-42e9-a64f-744efa2ef3a6 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:05.691148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22ef0a-655f-4f85-994a-1c73cfad0da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:05.692263 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22ef0a-655f-4f85-994a-1c73cfad0da0 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:05.692702 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22ef0a-655f-4f85-994a-1c73cfad0da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:05.693065 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22ef0a-655f-4f85-994a-1c73cfad0da0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:05.694435 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e22ef0a-655f-4f85-994a-1c73cfad0da0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:05.696713 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e22ef0a-655f-4f85-994a-1c73cfad0da0 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:05.711934 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 687/1382] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:05.713978 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3386c-9b63-40c0-be30-e7a2d026f97e admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:05.727774 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3386c-9b63-40c0-be30-e7a2d026f97e admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:05.735545 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1651a81-d87e-4d46-82ab-1a0498b23c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:05.736813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1651a81-d87e-4d46-82ab-1a0498b23c3b admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:05.737400 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1651a81-d87e-4d46-82ab-1a0498b23c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:05.737997 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1651a81-d87e-4d46-82ab-1a0498b23c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:05.739020 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1651a81-d87e-4d46-82ab-1a0498b23c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:05.741359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1651a81-d87e-4d46-82ab-1a0498b23c3b admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:05.778095 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22ef0a-655f-4f85-994a-1c73cfad0da0 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:05.781060 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8e22ef0a-655f-4f85-994a-1c73cfad0da0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:05.810771 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 696/1383] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:05.812743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22ef0a-655f-4f85-994a-1c73cfad0da0 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:05.814505 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1651a81-d87e-4d46-82ab-1a0498b23c3b admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:05.839750 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed4e8c-81f2-4adb-9a47-94856d1e4050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:05.840421 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed4e8c-81f2-4adb-9a47-94856d1e4050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:05.840927 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed4e8c-81f2-4adb-9a47-94856d1e4050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:05.841453 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed4e8c-81f2-4adb-9a47-94856d1e4050 admin admin] PATH_INFO: `/v3/projects/0ad4c46378764a42b925976d51254ab7` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:05.842880 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed4e8c-81f2-4adb-9a47-94856d1e4050 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ad4c46378764a42b925976d51254ab7)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:05.845745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1651a81-d87e-4d46-82ab-1a0498b23c3b admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:05.849779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed4e8c-81f2-4adb-9a47-94856d1e4050 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:05.859239 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e007988-d949-40cb-b9d4-9887465eed6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:05.859994 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e007988-d949-40cb-b9d4-9887465eed6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:05.860216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e007988-d949-40cb-b9d4-9887465eed6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:05.860510 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e007988-d949-40cb-b9d4-9887465eed6f admin admin] PATH_INFO: `/v3/projects/217ea8f8d30b424ea7d21fdbf40072bd/users/ece3a602ccd04435a2be578d25ee6753/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:05.919167 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e007988-d949-40cb-b9d4-9887465eed6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=217ea8f8d30b424ea7d21fdbf40072bd, user_id=ece3a602ccd04435a2be578d25ee6753, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:05.927154 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e007988-d949-40cb-b9d4-9887465eed6f admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:05.973427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 697/1384] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:05 2022] PUT /identity/v3/projects/217ea8f8d30b424ea7d21fdbf40072bd/users/ece3a602ccd04435a2be578d25ee6753/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:05.984818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f2c4f-bf42-4f36-a317-0ebd66afb1b4 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:05.985375 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f2c4f-bf42-4f36-a317-0ebd66afb1b4 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:05.985658 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f2c4f-bf42-4f36-a317-0ebd66afb1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:06.055676 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-ceed4e8c-81f2-4adb-9a47-94856d1e4050 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ad4c46378764a42b925976d51254ab7', 'request_id': 'req-ceed4e8c-81f2-4adb-9a47-94856d1e4050'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:06.079953 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-ceed4e8c-81f2-4adb-9a47-94856d1e4050 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ad4c46378764a42b925976d51254ab7', 'request_id': 'req-ceed4e8c-81f2-4adb-9a47-94856d1e4050'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:06.106361 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-ceed4e8c-81f2-4adb-9a47-94856d1e4050 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ad4c46378764a42b925976d51254ab7', 'request_id': 'req-ceed4e8c-81f2-4adb-9a47-94856d1e4050'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:06.144712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-659f2c4f-bf42-4f36-a317-0ebd66afb1b4 None None] MFA Rules not processed for user `ece3a602ccd04435a2be578d25ee6753`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:06.146735 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-659f2c4f-bf42-4f36-a317-0ebd66afb1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:06.176369 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-ceed4e8c-81f2-4adb-9a47-94856d1e4050 admin admin] The token cache is being invalidate because project 0ad4c46378764a42b925976d51254ab7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:06.178515 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-ceed4e8c-81f2-4adb-9a47-94856d1e4050 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:06.212316 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-ceed4e8c-81f2-4adb-9a47-94856d1e4050 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:06.225860 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 688/1385] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:05 2022] DELETE /identity/v3/projects/0ad4c46378764a42b925976d51254ab7 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:08:06.228109 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36564ed-29e8-4711-a269-cfd0c57e1295 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:06.230058 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36564ed-29e8-4711-a269-cfd0c57e1295 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:06.230668 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36564ed-29e8-4711-a269-cfd0c57e1295 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:06.232936 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 689/1386] 10.222.0.13 () {54 vars in 912 bytes} [Sun Oct 23 00:08:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:08:06.245671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10512d-c087-4bd5-a660-d318b98d7f8f None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:06.246215 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10512d-c087-4bd5-a660-d318b98d7f8f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:06.249514 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10512d-c087-4bd5-a660-d318b98d7f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:06.322383 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-5d10512d-c087-4bd5-a660-d318b98d7f8f None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:06.324318 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-5d10512d-c087-4bd5-a660-d318b98d7f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:06.344227 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 698/1387] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:06.345757 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbec20e-9ed4-4618-ae62-7c9a70902b92 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:06.346046 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbec20e-9ed4-4618-ae62-7c9a70902b92 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:06.346325 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbec20e-9ed4-4618-ae62-7c9a70902b92 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:06.347942 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 699/1388] 10.222.0.13 () {54 vars in 912 bytes} [Sun Oct 23 00:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:08:06.350215 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee47d8d-d08b-48bf-abb7-2996ce07d007 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:06.350503 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee47d8d-d08b-48bf-abb7-2996ce07d007 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:06.350776 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee47d8d-d08b-48bf-abb7-2996ce07d007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:06.443261 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 690/1389] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:06.443261 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10512d-c087-4bd5-a660-d318b98d7f8f None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:06.443261 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-5d10512d-c087-4bd5-a660-d318b98d7f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:06.444560 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-3ee47d8d-d08b-48bf-abb7-2996ce07d007 None None] MFA Rules not processed for user `a09d45d080bc461c88f38ff281b75b06`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:06.445769 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-3ee47d8d-d08b-48bf-abb7-2996ce07d007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:06.458418 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10512d-c087-4bd5-a660-d318b98d7f8f None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:06.472084 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a86782-fb21-4549-93ab-7957ce3a1858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:06.475166 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a86782-fb21-4549-93ab-7957ce3a1858 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:06.475634 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a86782-fb21-4549-93ab-7957ce3a1858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:06.476009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a86782-fb21-4549-93ab-7957ce3a1858 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:06.476871 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a86782-fb21-4549-93ab-7957ce3a1858 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:06.482144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a86782-fb21-4549-93ab-7957ce3a1858 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:06.520838 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 691/1390] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:06 2022] POST /identity/v3/projects => generated 363 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:06.522686 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca13e8-c437-47c9-a150-d03826dd8ee0 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:06.523071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca13e8-c437-47c9-a150-d03826dd8ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:06.523710 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca13e8-c437-47c9-a150-d03826dd8ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:06.533191 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:06.550462 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-6bca13e8-c437-47c9-a150-d03826dd8ee0 None None] MFA Rules not processed for user `a09d45d080bc461c88f38ff281b75b06`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:06.553161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-6bca13e8-c437-47c9-a150-d03826dd8ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:06.581627 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 700/1391] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 00:08:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:06.583056 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4d11d-0762-4363-b9ac-83526c942f34 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:06.583406 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4d11d-0762-4363-b9ac-83526c942f34 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:06.583687 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4d11d-0762-4363-b9ac-83526c942f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:06.601212 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:06.639734 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-eff4d11d-0762-4363-b9ac-83526c942f34 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:06.644015 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-eff4d11d-0762-4363-b9ac-83526c942f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:06.741006 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 692/1392] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 00:08:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:06.742090 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca13e8-c437-47c9-a150-d03826dd8ee0 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:06.808591 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 701/1393] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:06.813365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4d11d-0762-4363-b9ac-83526c942f34 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:06.814217 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-eff4d11d-0762-4363-b9ac-83526c942f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:06.830126 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4d11d-0762-4363-b9ac-83526c942f34 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:06.839105 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45dac56-dbc4-4f7a-a630-1d942b955f84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:06.839726 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45dac56-dbc4-4f7a-a630-1d942b955f84 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:06.840136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45dac56-dbc4-4f7a-a630-1d942b955f84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:06.840412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45dac56-dbc4-4f7a-a630-1d942b955f84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:06.850040 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca13e8-c437-47c9-a150-d03826dd8ee0 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:06.855038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61da391d-6d16-48c0-a248-1a3dda3ea797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:06.856428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da391d-6d16-48c0-a248-1a3dda3ea797 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:06.856909 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da391d-6d16-48c0-a248-1a3dda3ea797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:06.857351 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61da391d-6d16-48c0-a248-1a3dda3ea797 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:06.857512 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45dac56-dbc4-4f7a-a630-1d942b955f84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:06.859502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da391d-6d16-48c0-a248-1a3dda3ea797 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:06.860351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45dac56-dbc4-4f7a-a630-1d942b955f84 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:06.861954 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61da391d-6d16-48c0-a248-1a3dda3ea797 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:06.881151 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 702/1394] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:06.882745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45dac56-dbc4-4f7a-a630-1d942b955f84 service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:06.897952 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45dac56-dbc4-4f7a-a630-1d942b955f84 service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:06.903231 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67a35b-391c-4d42-9694-7c63b8faf607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:06.904862 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67a35b-391c-4d42-9694-7c63b8faf607 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:06.905320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67a35b-391c-4d42-9694-7c63b8faf607 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:06.905663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67a35b-391c-4d42-9694-7c63b8faf607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:06.913208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 693/1395] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:06 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:06.914983 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8670c8-70f3-47ab-95e8-bd11a55007ac None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:06.916869 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8670c8-70f3-47ab-95e8-bd11a55007ac None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:06.917312 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8670c8-70f3-47ab-95e8-bd11a55007ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:06.918187 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67a35b-391c-4d42-9694-7c63b8faf607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:06.920551 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67a35b-391c-4d42-9694-7c63b8faf607 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:06.933794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 703/1396] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:06.935935 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67a35b-391c-4d42-9694-7c63b8faf607 service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:06.953487 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67a35b-391c-4d42-9694-7c63b8faf607 service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:06.958657 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fa958-0415-497e-be27-5643dba87c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:06.959035 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-6f8670c8-70f3-47ab-95e8-bd11a55007ac None None] MFA Rules not processed for user `8b4ef165353f4f1189400c005519c2ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:06.959411 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fa958-0415-497e-be27-5643dba87c6a admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:06.959887 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fa958-0415-497e-be27-5643dba87c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:06.960237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fa958-0415-497e-be27-5643dba87c6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:06.960346 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-6f8670c8-70f3-47ab-95e8-bd11a55007ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:06.961173 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458fa958-0415-497e-be27-5643dba87c6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:08:06.961588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458fa958-0415-497e-be27-5643dba87c6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:06.963559 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458fa958-0415-497e-be27-5643dba87c6a admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:06.979222 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 704/1397] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:08:06.981451 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fa958-0415-497e-be27-5643dba87c6a admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:06.998848 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fa958-0415-497e-be27-5643dba87c6a admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:07.005495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30c479-113c-4898-86af-ab411088ca96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:07.006285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30c479-113c-4898-86af-ab411088ca96 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:07.006724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30c479-113c-4898-86af-ab411088ca96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:07.007128 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d30c479-113c-4898-86af-ab411088ca96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:07.008191 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d30c479-113c-4898-86af-ab411088ca96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:07.009949 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d30c479-113c-4898-86af-ab411088ca96 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:07.032255 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 705/1398] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:07.033462 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30c479-113c-4898-86af-ab411088ca96 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:07.046989 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d30c479-113c-4898-86af-ab411088ca96 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:07.060478 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3a233-52b6-4919-a155-520c75b70868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:07.060478 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3a233-52b6-4919-a155-520c75b70868 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:07.060478 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3a233-52b6-4919-a155-520c75b70868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:07.060478 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3a233-52b6-4919-a155-520c75b70868 admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:07.061147 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3a233-52b6-4919-a155-520c75b70868 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:07.062784 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3a233-52b6-4919-a155-520c75b70868 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:07.097301 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 694/1399] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:06 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:07.098490 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8670c8-70f3-47ab-95e8-bd11a55007ac None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:07.126567 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8670c8-70f3-47ab-95e8-bd11a55007ac None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:07.134743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150c700-48bc-4915-a25b-65dda185ac49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:07.135940 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150c700-48bc-4915-a25b-65dda185ac49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:07.136265 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150c700-48bc-4915-a25b-65dda185ac49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:07.136581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8150c700-48bc-4915-a25b-65dda185ac49 admin admin] PATH_INFO: `/v3/projects/3c27369a433e4d9bb209c24a6980ca1c/users/101926168e2940868d275895a3bf5eee/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:07.141941 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 706/1400] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:07 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:07.146123 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3a233-52b6-4919-a155-520c75b70868 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:07.158951 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150c700-48bc-4915-a25b-65dda185ac49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c27369a433e4d9bb209c24a6980ca1c, user_id=101926168e2940868d275895a3bf5eee, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:07.161917 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8150c700-48bc-4915-a25b-65dda185ac49 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:07.180522 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 695/1401] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:07 2022] PUT /identity/v3/projects/3c27369a433e4d9bb209c24a6980ca1c/users/101926168e2940868d275895a3bf5eee/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:07.184666 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150c700-48bc-4915-a25b-65dda185ac49 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:07.226580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:07.274989 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3a233-52b6-4919-a155-520c75b70868 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:07.291038 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0e7f0-6636-4256-a2ab-91f616a6f1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:07.291668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0e7f0-6636-4256-a2ab-91f616a6f1e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:07.291948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0e7f0-6636-4256-a2ab-91f616a6f1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:07.292221 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0e7f0-6636-4256-a2ab-91f616a6f1e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:07.292967 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0e7f0-6636-4256-a2ab-91f616a6f1e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:07.295233 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a0e7f0-6636-4256-a2ab-91f616a6f1e4 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:07.296893 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8150c700-48bc-4915-a25b-65dda185ac49 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:07.304633 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386bf74-343e-4f8f-8085-e0764f6c5aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:07.305505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386bf74-343e-4f8f-8085-e0764f6c5aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:07.305856 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386bf74-343e-4f8f-8085-e0764f6c5aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:07.306206 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386bf74-343e-4f8f-8085-e0764f6c5aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:07.307242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3386bf74-343e-4f8f-8085-e0764f6c5aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:07.309105 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3386bf74-343e-4f8f-8085-e0764f6c5aa2 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:07.328246 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 696/1402] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:07.330806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386bf74-343e-4f8f-8085-e0764f6c5aa2 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:07.345450 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 707/1403] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:07 2022] POST /identity/v3/users => generated 294 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:07.349824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0e7f0-6636-4256-a2ab-91f616a6f1e4 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:07.352670 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386bf74-343e-4f8f-8085-e0764f6c5aa2 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:07.365273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08ee75f-7ac3-4eaa-878e-623b7cbb1f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:07.367128 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ee75f-7ac3-4eaa-878e-623b7cbb1f8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:07.370262 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0e7f0-6636-4256-a2ab-91f616a6f1e4 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:07.371565 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ee75f-7ac3-4eaa-878e-623b7cbb1f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:07.376170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ee75f-7ac3-4eaa-878e-623b7cbb1f8c admin admin] PATH_INFO: `/v3/projects/3c27369a433e4d9bb209c24a6980ca1c/users/101926168e2940868d275895a3bf5eee/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:07.378922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c2268b-c2f9-4859-b7ee-c1db1f510443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:07.379794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2268b-c2f9-4859-b7ee-c1db1f510443 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:07.384163 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2268b-c2f9-4859-b7ee-c1db1f510443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:07.384664 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2268b-c2f9-4859-b7ee-c1db1f510443 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:07.384903 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08ee75f-7ac3-4eaa-878e-623b7cbb1f8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c27369a433e4d9bb209c24a6980ca1c, user_id=101926168e2940868d275895a3bf5eee, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:07.385854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c2268b-c2f9-4859-b7ee-c1db1f510443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:07.387787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c2268b-c2f9-4859-b7ee-c1db1f510443 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:07.388114 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08ee75f-7ac3-4eaa-878e-623b7cbb1f8c admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:07.408263 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 708/1404] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:07.410601 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 697/1405] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:07 2022] PUT /identity/v3/projects/3c27369a433e4d9bb209c24a6980ca1c/users/101926168e2940868d275895a3bf5eee/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:07.411888 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c2268b-c2f9-4859-b7ee-c1db1f510443 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:07.423210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fe379-8272-44f4-a1e5-499955c747ec None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:07.423870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fe379-8272-44f4-a1e5-499955c747ec None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:07.424370 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fe379-8272-44f4-a1e5-499955c747ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:07.444321 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c2268b-c2f9-4859-b7ee-c1db1f510443 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:07.460237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:07.474932 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-913fe379-8272-44f4-a1e5-499955c747ec None None] MFA Rules not processed for user `101926168e2940868d275895a3bf5eee`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:07.476271 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-913fe379-8272-44f4-a1e5-499955c747ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:07.586381 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 698/1406] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:07 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:08:07.597079 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fe379-8272-44f4-a1e5-499955c747ec None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:07.599246 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-913fe379-8272-44f4-a1e5-499955c747ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:07.630064 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f640f-8da3-4b8b-a1fb-eaa5c8870164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:07.631339 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f640f-8da3-4b8b-a1fb-eaa5c8870164 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:07.631339 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f640f-8da3-4b8b-a1fb-eaa5c8870164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:07.631547 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f640f-8da3-4b8b-a1fb-eaa5c8870164 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:07.633107 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f640f-8da3-4b8b-a1fb-eaa5c8870164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:07.637064 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f640f-8da3-4b8b-a1fb-eaa5c8870164 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:07.666226 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 709/1407] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:08:07.669120 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f640f-8da3-4b8b-a1fb-eaa5c8870164 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:07.697219 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f640f-8da3-4b8b-a1fb-eaa5c8870164 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:07.709158 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aad5c2-2cf7-4c78-bfd6-ced067ba4a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:07.709978 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aad5c2-2cf7-4c78-bfd6-ced067ba4a4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:07.710294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aad5c2-2cf7-4c78-bfd6-ced067ba4a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:07.715105 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aad5c2-2cf7-4c78-bfd6-ced067ba4a4d admin admin] PATH_INFO: `/v3/projects/cb9121f281484ad89e59bf20e9e0a276/users/9f11a38793d94bfd815581f14b07ac6b/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:07.738028 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fe379-8272-44f4-a1e5-499955c747ec None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:07.743928 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8ad67-6532-43aa-a319-6e40656d27bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:07.743928 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8ad67-6532-43aa-a319-6e40656d27bf service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:07.743928 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8ad67-6532-43aa-a319-6e40656d27bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:07.743928 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8ad67-6532-43aa-a319-6e40656d27bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:07.789768 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8ad67-6532-43aa-a319-6e40656d27bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:07.791209 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8ad67-6532-43aa-a319-6e40656d27bf service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:07.798319 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aad5c2-2cf7-4c78-bfd6-ced067ba4a4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb9121f281484ad89e59bf20e9e0a276, user_id=9f11a38793d94bfd815581f14b07ac6b, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:07.815481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aad5c2-2cf7-4c78-bfd6-ced067ba4a4d admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:07.825189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 699/1408] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:07 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:08:07.878412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 710/1409] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:07 2022] PUT /identity/v3/projects/cb9121f281484ad89e59bf20e9e0a276/users/9f11a38793d94bfd815581f14b07ac6b/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:07.890248 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f9dfc-4707-4a23-b216-678c08b6d3af None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:07.890547 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f9dfc-4707-4a23-b216-678c08b6d3af None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:07.890827 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f9dfc-4707-4a23-b216-678c08b6d3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:07.895848 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aad5c2-2cf7-4c78-bfd6-ced067ba4a4d admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:07.898809 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b3aad5c2-2cf7-4c78-bfd6-ced067ba4a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:07.951098 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-940f9dfc-4707-4a23-b216-678c08b6d3af None None] MFA Rules not processed for user `9f11a38793d94bfd815581f14b07ac6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:07.952948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-940f9dfc-4707-4a23-b216-678c08b6d3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:08.019960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:08.068062 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aad5c2-2cf7-4c78-bfd6-ced067ba4a4d admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:08.101037 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9574913-b543-4ebc-91df-de5a9895c0d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24108b6355414f64ba5e7bf84011eee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:08.101621 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9574913-b543-4ebc-91df-de5a9895c0d7 service placement] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:08.104203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9574913-b543-4ebc-91df-de5a9895c0d7 service placement] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:08.105562 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9574913-b543-4ebc-91df-de5a9895c0d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:08.110959 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e9574913-b543-4ebc-91df-de5a9895c0d7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:08.115710 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 700/1410] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:08.117434 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca4b04-5b20-4253-8227-7a1f6f7e3ecd None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:08.117732 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca4b04-5b20-4253-8227-7a1f6f7e3ecd None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:08.118015 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca4b04-5b20-4253-8227-7a1f6f7e3ecd None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:08.127647 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 701/1411] 10.222.0.13 () {54 vars in 961 bytes} [Sun Oct 23 00:08:08 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:08:08.128640 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca4b04-5b20-4253-8227-7a1f6f7e3ecd None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:08.165568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9574913-b543-4ebc-91df-de5a9895c0d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:08.167897 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9574913-b543-4ebc-91df-de5a9895c0d7 service placement] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:08.232971 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 711/1412] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:08:08.236498 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075b41b-9c73-41bf-9052-abfc920ebfd8 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:08.239076 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075b41b-9c73-41bf-9052-abfc920ebfd8 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:08.239536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5075b41b-9c73-41bf-9052-abfc920ebfd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:08.246067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca4b04-5b20-4253-8227-7a1f6f7e3ecd None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:08.261006 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca4138-c378-4ba5-ae10-6a3622e160a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:08.261759 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca4138-c378-4ba5-ae10-6a3622e160a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:08.262153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca4138-c378-4ba5-ae10-6a3622e160a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:08.262556 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca4138-c378-4ba5-ae10-6a3622e160a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:08.269466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca4138-c378-4ba5-ae10-6a3622e160a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:08.269466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca4138-c378-4ba5-ae10-6a3622e160a0 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:08.316504 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-5075b41b-9c73-41bf-9052-abfc920ebfd8 None None] MFA Rules not processed for user `24108b6355414f64ba5e7bf84011eee6`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:08.319305 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5075b41b-9c73-41bf-9052-abfc920ebfd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:08.338534 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 712/1413] 10.222.0.13 () {58 vars in 1058 bytes} [Sun Oct 23 00:08:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:08.341892 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 702/1414] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:08 2022] POST /identity/v3/projects => generated 385 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:08.344706 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075b41b-9c73-41bf-9052-abfc920ebfd8 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:08.350731 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca4138-c378-4ba5-ae10-6a3622e160a0 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:08.366161 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5075b41b-9c73-41bf-9052-abfc920ebfd8 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:08.590096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca4138-c378-4ba5-ae10-6a3622e160a0 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:08.605211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505d641-004e-4816-aaf1-226415250d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:08.605849 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505d641-004e-4816-aaf1-226415250d80 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:08.606136 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505d641-004e-4816-aaf1-226415250d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:08.608945 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505d641-004e-4816-aaf1-226415250d80 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:08.610062 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18015922-807b-47f9-9a90-672d6224e127 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24108b6355414f64ba5e7bf84011eee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:08.610689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18015922-807b-47f9-9a90-672d6224e127 service placement] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:08.610971 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18015922-807b-47f9-9a90-672d6224e127 service placement] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:08.611571 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505d641-004e-4816-aaf1-226415250d80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:08.613592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505d641-004e-4816-aaf1-226415250d80 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:08.615002 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18015922-807b-47f9-9a90-672d6224e127 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:08.648648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:08.682818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18015922-807b-47f9-9a90-672d6224e127 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:08.690678 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18015922-807b-47f9-9a90-672d6224e127 service placement] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:08.748049 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 703/1415] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:08 2022] POST /identity/v3/users => generated 293 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:08.762290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e97900-73fb-4094-8f41-816880857a2f None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:08.771192 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 713/1416] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:08.772351 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e97900-73fb-4094-8f41-816880857a2f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:08.773377 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e97900-73fb-4094-8f41-816880857a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:08.781612 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18015922-807b-47f9-9a90-672d6224e127 service placement] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:08.857187 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18015922-807b-47f9-9a90-672d6224e127 service placement] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:08.879098 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-f3e97900-73fb-4094-8f41-816880857a2f None None] MFA Rules not processed for user `f95f4fc0d5c5428c96545f543139489b`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:08.880508 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f3e97900-73fb-4094-8f41-816880857a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:08.884790 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d537ff-69dc-43dc-b3cc-3974372e0342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:08.886843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d537ff-69dc-43dc-b3cc-3974372e0342 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:08.887291 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d537ff-69dc-43dc-b3cc-3974372e0342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:08.887711 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d537ff-69dc-43dc-b3cc-3974372e0342 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:08.888896 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d537ff-69dc-43dc-b3cc-3974372e0342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:08.892909 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d537ff-69dc-43dc-b3cc-3974372e0342 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:08.963102 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 714/1417] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:08.965223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d537ff-69dc-43dc-b3cc-3974372e0342 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:09.002405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d537ff-69dc-43dc-b3cc-3974372e0342 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:09.013232 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:09.013426 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6478d63-4513-4360-9759-71821f2c3bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:09.016536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6478d63-4513-4360-9759-71821f2c3bf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:09.016536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6478d63-4513-4360-9759-71821f2c3bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:09.016536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6478d63-4513-4360-9759-71821f2c3bf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:09.017343 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6478d63-4513-4360-9759-71821f2c3bf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:09.024869 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6478d63-4513-4360-9759-71821f2c3bf7 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:09.062387 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 715/1418] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:09.066071 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6478d63-4513-4360-9759-71821f2c3bf7 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:09.087946 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 704/1419] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:09.094301 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6478d63-4513-4360-9759-71821f2c3bf7 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:09.124804 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a77b5-8cdd-4c77-8b72-98b5aa74c502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:09.125995 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a77b5-8cdd-4c77-8b72-98b5aa74c502 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:09.128064 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a77b5-8cdd-4c77-8b72-98b5aa74c502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:09.128669 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a77b5-8cdd-4c77-8b72-98b5aa74c502 admin admin] PATH_INFO: `/v3/projects/679f88d0ca8b4a719a079ed0d8daf83a/users/69bec4d01e4243de98e18350697b7955/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:09.204749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a77b5-8cdd-4c77-8b72-98b5aa74c502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=679f88d0ca8b4a719a079ed0d8daf83a, user_id=69bec4d01e4243de98e18350697b7955, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:09.214001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a77b5-8cdd-4c77-8b72-98b5aa74c502 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:09.222457 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e97900-73fb-4094-8f41-816880857a2f None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:09.246106 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e97900-73fb-4094-8f41-816880857a2f None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:09.258150 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 716/1420] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:09 2022] PUT /identity/v3/projects/679f88d0ca8b4a719a079ed0d8daf83a/users/69bec4d01e4243de98e18350697b7955/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:09.270718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cafb96-2e8a-4b74-b943-f1fce909e68b None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:09.271008 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cafb96-2e8a-4b74-b943-f1fce909e68b None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:09.271358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cafb96-2e8a-4b74-b943-f1fce909e68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:09.370537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-a2cafb96-2e8a-4b74-b943-f1fce909e68b None None] MFA Rules not processed for user `69bec4d01e4243de98e18350697b7955`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:09.373520 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a2cafb96-2e8a-4b74-b943-f1fce909e68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:09.461815 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c75f7-6a4b-4a9e-901d-832db058c59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:09.463386 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c75f7-6a4b-4a9e-901d-832db058c59e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:09.463820 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c75f7-6a4b-4a9e-901d-832db058c59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:09.464238 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c75f7-6a4b-4a9e-901d-832db058c59e admin admin] PATH_INFO: `/v3/users/ece3a602ccd04435a2be578d25ee6753` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:09.467237 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c75f7-6a4b-4a9e-901d-832db058c59e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ece3a602ccd04435a2be578d25ee6753)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:09.469994 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c75f7-6a4b-4a9e-901d-832db058c59e admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:09.646404 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-8a3c75f7-6a4b-4a9e-901d-832db058c59e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ece3a602ccd04435a2be578d25ee6753'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:09.647444 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-8a3c75f7-6a4b-4a9e-901d-832db058c59e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ece3a602ccd04435a2be578d25ee6753'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:09.648570 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 717/1421] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:08:09.652292 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cafb96-2e8a-4b74-b943-f1fce909e68b None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:09.667607 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-8a3c75f7-6a4b-4a9e-901d-832db058c59e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ece3a602ccd04435a2be578d25ee6753'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:09.672823 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cafb96-2e8a-4b74-b943-f1fce909e68b None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:09.684701 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f19b9a-7c14-48b8-92b2-b2979848e109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:09.685408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f19b9a-7c14-48b8-92b2-b2979848e109 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:09.685891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f19b9a-7c14-48b8-92b2-b2979848e109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:09.686324 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f19b9a-7c14-48b8-92b2-b2979848e109 admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:09.693227 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f19b9a-7c14-48b8-92b2-b2979848e109 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:09.700216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f19b9a-7c14-48b8-92b2-b2979848e109 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:09.709789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-8a3c75f7-6a4b-4a9e-901d-832db058c59e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ece3a602ccd04435a2be578d25ee6753'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:09.727086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-8a3c75f7-6a4b-4a9e-901d-832db058c59e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ece3a602ccd04435a2be578d25ee6753'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:09.739217 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 705/1422] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:09 2022] DELETE /identity/v3/users/ece3a602ccd04435a2be578d25ee6753 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:08:09.743892 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b02f33-e373-4b02-9f83-0254abda25d5 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:09.744570 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b02f33-e373-4b02-9f83-0254abda25d5 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:09.745986 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b02f33-e373-4b02-9f83-0254abda25d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:09.787911 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 718/1423] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:09 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:09.794064 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f19b9a-7c14-48b8-92b2-b2979848e109 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:09.799764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-64f19b9a-7c14-48b8-92b2-b2979848e109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:09.811180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-81b02f33-e373-4b02-9f83-0254abda25d5 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:09.812557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-81b02f33-e373-4b02-9f83-0254abda25d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:09.973565 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f19b9a-7c14-48b8-92b2-b2979848e109 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:09.990305 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36c719-5a5a-49c7-896d-538c2f7a60a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:09.991137 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36c719-5a5a-49c7-896d-538c2f7a60a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:09.991554 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36c719-5a5a-49c7-896d-538c2f7a60a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:09.991974 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36c719-5a5a-49c7-896d-538c2f7a60a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:09.992791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36c719-5a5a-49c7-896d-538c2f7a60a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:09.996831 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36c719-5a5a-49c7-896d-538c2f7a60a9 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:10.073356 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 719/1424] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:09 2022] POST /identity/v3/users => generated 299 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:10.080979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36c719-5a5a-49c7-896d-538c2f7a60a9 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:10.084166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-2c36c719-5a5a-49c7-896d-538c2f7a60a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:10.122015 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:10.142071 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36c719-5a5a-49c7-896d-538c2f7a60a9 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:10.151697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 706/1425] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:10.156066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b02f33-e373-4b02-9f83-0254abda25d5 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:10.166197 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ce061-f99f-4bec-8b2d-401de29688c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:10.167092 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ce061-f99f-4bec-8b2d-401de29688c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:10.167558 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ce061-f99f-4bec-8b2d-401de29688c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:10.167956 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ce061-f99f-4bec-8b2d-401de29688c1 admin admin] PATH_INFO: `/v3/users/ea5a0f5f9d024d299b5a867681aa474b` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:10.170795 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ce061-f99f-4bec-8b2d-401de29688c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea5a0f5f9d024d299b5a867681aa474b)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:10.176682 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:10.197484 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b02f33-e373-4b02-9f83-0254abda25d5 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:10.209468 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ce061-f99f-4bec-8b2d-401de29688c1 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:10.217203 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff1ebb-edc8-4580-8839-a62ec1bc10a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:10.218227 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff1ebb-edc8-4580-8839-a62ec1bc10a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:10.218509 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff1ebb-edc8-4580-8839-a62ec1bc10a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:10.218791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff1ebb-edc8-4580-8839-a62ec1bc10a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:10.219442 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff1ebb-edc8-4580-8839-a62ec1bc10a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:10.242008 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff1ebb-edc8-4580-8839-a62ec1bc10a2 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:10.300183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 707/1426] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:10.300183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff1ebb-edc8-4580-8839-a62ec1bc10a2 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:10.307305 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-85ff1ebb-edc8-4580-8839-a62ec1bc10a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:10.379577 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4c2ce061-f99f-4bec-8b2d-401de29688c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea5a0f5f9d024d299b5a867681aa474b'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:10.398122 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4c2ce061-f99f-4bec-8b2d-401de29688c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea5a0f5f9d024d299b5a867681aa474b'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:10.431656 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4c2ce061-f99f-4bec-8b2d-401de29688c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea5a0f5f9d024d299b5a867681aa474b'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:10.433965 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4c2ce061-f99f-4bec-8b2d-401de29688c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea5a0f5f9d024d299b5a867681aa474b'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:10.435275 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4c2ce061-f99f-4bec-8b2d-401de29688c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea5a0f5f9d024d299b5a867681aa474b'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:10.449972 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff1ebb-edc8-4580-8839-a62ec1bc10a2 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:10.464800 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fbad34-99a4-4274-9cdd-996a1b7f1e83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:10.465890 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbad34-99a4-4274-9cdd-996a1b7f1e83 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:10.465890 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbad34-99a4-4274-9cdd-996a1b7f1e83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:10.466124 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbad34-99a4-4274-9cdd-996a1b7f1e83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:10.466958 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 720/1427] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:10 2022] DELETE /identity/v3/users/ea5a0f5f9d024d299b5a867681aa474b => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:10.468123 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ce061-f99f-4bec-8b2d-401de29688c1 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:10.469893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-4c2ce061-f99f-4bec-8b2d-401de29688c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:10.471807 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-37fbad34-99a4-4274-9cdd-996a1b7f1e83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:10.522746 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fbad34-99a4-4274-9cdd-996a1b7f1e83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:10.530377 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fbad34-99a4-4274-9cdd-996a1b7f1e83 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:10.563584 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 708/1428] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:10.565406 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00bf27-faba-433d-bd6f-bf320f837c8a None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:10.565853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00bf27-faba-433d-bd6f-bf320f837c8a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:10.566289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00bf27-faba-433d-bd6f-bf320f837c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:10.626818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-cd00bf27-faba-433d-bd6f-bf320f837c8a None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:10.629675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-cd00bf27-faba-433d-bd6f-bf320f837c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:10.712440 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ce061-f99f-4bec-8b2d-401de29688c1 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:10.727936 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bde7e8-1dcc-4340-8f58-1f7d58e63167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:10.728676 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bde7e8-1dcc-4340-8f58-1f7d58e63167 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:10.728967 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bde7e8-1dcc-4340-8f58-1f7d58e63167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:10.729244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bde7e8-1dcc-4340-8f58-1f7d58e63167 admin admin] PATH_INFO: `/v3/projects/192e5a0e83e04615a4fedc0c22e4e00b/users/921ca365cef74dec8cc6fbb3aa91e036/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:10.769452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bde7e8-1dcc-4340-8f58-1f7d58e63167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=192e5a0e83e04615a4fedc0c22e4e00b, user_id=921ca365cef74dec8cc6fbb3aa91e036, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:10.772449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bde7e8-1dcc-4340-8f58-1f7d58e63167 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:10.809753 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 721/1429] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:10 2022] PUT /identity/v3/projects/192e5a0e83e04615a4fedc0c22e4e00b/users/921ca365cef74dec8cc6fbb3aa91e036/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:10.811554 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bde7e8-1dcc-4340-8f58-1f7d58e63167 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:10.822942 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 709/1430] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:10.858082 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00bf27-faba-433d-bd6f-bf320f837c8a None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:10.861474 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-cd00bf27-faba-433d-bd6f-bf320f837c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:10.945092 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bde7e8-1dcc-4340-8f58-1f7d58e63167 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:10.951519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63cf24-12e4-45c8-8f0c-df311b4dddf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:10.952629 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63cf24-12e4-45c8-8f0c-df311b4dddf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:10.953208 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63cf24-12e4-45c8-8f0c-df311b4dddf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:10.953814 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63cf24-12e4-45c8-8f0c-df311b4dddf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:10.954603 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a63cf24-12e4-45c8-8f0c-df311b4dddf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:10.957137 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a63cf24-12e4-45c8-8f0c-df311b4dddf3 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:10.958842 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:10.977471 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 722/1431] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:08:10.980025 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63cf24-12e4-45c8-8f0c-df311b4dddf3 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:10.998063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63cf24-12e4-45c8-8f0c-df311b4dddf3 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:11.004178 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aac839-fb7b-445b-8bad-031b085f23eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:11.005440 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aac839-fb7b-445b-8bad-031b085f23eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:11.005836 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aac839-fb7b-445b-8bad-031b085f23eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:11.006321 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aac839-fb7b-445b-8bad-031b085f23eb admin admin] PATH_INFO: `/v3/projects/192e5a0e83e04615a4fedc0c22e4e00b/users/921ca365cef74dec8cc6fbb3aa91e036/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:11.008764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aac839-fb7b-445b-8bad-031b085f23eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=192e5a0e83e04615a4fedc0c22e4e00b, user_id=921ca365cef74dec8cc6fbb3aa91e036, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:11.011945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aac839-fb7b-445b-8bad-031b085f23eb admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:11.032287 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 723/1432] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:10 2022] PUT /identity/v3/projects/192e5a0e83e04615a4fedc0c22e4e00b/users/921ca365cef74dec8cc6fbb3aa91e036/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:11.042494 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00bf27-faba-433d-bd6f-bf320f837c8a None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:11.044087 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5b760-8468-49b2-bcfd-8f22789f9f08 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:11.044787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5b760-8468-49b2-bcfd-8f22789f9f08 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:11.045233 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5b760-8468-49b2-bcfd-8f22789f9f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:11.126825 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-64b5b760-8468-49b2-bcfd-8f22789f9f08 None None] MFA Rules not processed for user `921ca365cef74dec8cc6fbb3aa91e036`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:11.128271 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-64b5b760-8468-49b2-bcfd-8f22789f9f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:11.193960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4835131-52fe-4fc4-9b95-57a6c990a106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:11.195938 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4835131-52fe-4fc4-9b95-57a6c990a106 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:11.196204 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4835131-52fe-4fc4-9b95-57a6c990a106 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:11.198767 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4835131-52fe-4fc4-9b95-57a6c990a106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:11.231539 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:11.359029 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 724/1433] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:11 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:11.378660 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5b760-8468-49b2-bcfd-8f22789f9f08 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:11.385039 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-64b5b760-8468-49b2-bcfd-8f22789f9f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:11.601540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4835131-52fe-4fc4-9b95-57a6c990a106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:11.603632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4835131-52fe-4fc4-9b95-57a6c990a106 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:11.650969 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 710/1434] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 794 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:11.670606 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4835131-52fe-4fc4-9b95-57a6c990a106 service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:11.676104 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d4835131-52fe-4fc4-9b95-57a6c990a106 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:11.777024 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5b760-8468-49b2-bcfd-8f22789f9f08 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:11.793565 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd80e6-0279-41bd-ba64-80a3c17ad17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:11.794477 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd80e6-0279-41bd-ba64-80a3c17ad17c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:11.795010 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd80e6-0279-41bd-ba64-80a3c17ad17c admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:11.795484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd80e6-0279-41bd-ba64-80a3c17ad17c admin admin] PATH_INFO: `/v3/users/40e1a487cab9418480172aa1f32d7caf` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:11.796992 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd80e6-0279-41bd-ba64-80a3c17ad17c admin admin] RBAC: Authorizing `identity:delete_user(user_id=40e1a487cab9418480172aa1f32d7caf)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:11.799144 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd80e6-0279-41bd-ba64-80a3c17ad17c admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:11.922884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4835131-52fe-4fc4-9b95-57a6c990a106 service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:11.934476 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-a0bd80e6-0279-41bd-ba64-80a3c17ad17c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40e1a487cab9418480172aa1f32d7caf'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:11.960116 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-a0bd80e6-0279-41bd-ba64-80a3c17ad17c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40e1a487cab9418480172aa1f32d7caf'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:11.961615 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81edda27-5ec3-4679-b43e-a84783e3301b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:11.962276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edda27-5ec3-4679-b43e-a84783e3301b service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:11.970324 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edda27-5ec3-4679-b43e-a84783e3301b service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:11.970762 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edda27-5ec3-4679-b43e-a84783e3301b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:11.985839 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:11.992256 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-a0bd80e6-0279-41bd-ba64-80a3c17ad17c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40e1a487cab9418480172aa1f32d7caf'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:11.996616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-a0bd80e6-0279-41bd-ba64-80a3c17ad17c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40e1a487cab9418480172aa1f32d7caf'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:11.998708 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-a0bd80e6-0279-41bd-ba64-80a3c17ad17c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40e1a487cab9418480172aa1f32d7caf'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:12.029168 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 725/1435] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:11 2022] DELETE /identity/v3/users/40e1a487cab9418480172aa1f32d7caf => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:12.032380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd80e6-0279-41bd-ba64-80a3c17ad17c admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:12.035630 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a0bd80e6-0279-41bd-ba64-80a3c17ad17c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:12.065024 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81edda27-5ec3-4679-b43e-a84783e3301b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:12.072021 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81edda27-5ec3-4679-b43e-a84783e3301b service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:12.123800 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 711/1436] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:12.125603 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1e663-e1b2-4bb1-82a6-7f1ef0ae5152 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:12.129241 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1e663-e1b2-4bb1-82a6-7f1ef0ae5152 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:12.130219 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1e663-e1b2-4bb1-82a6-7f1ef0ae5152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:12.208815 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-45b1e663-e1b2-4bb1-82a6-7f1ef0ae5152 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:12.210119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-45b1e663-e1b2-4bb1-82a6-7f1ef0ae5152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:12.259921 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd80e6-0279-41bd-ba64-80a3c17ad17c admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:12.289272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d9b99-e202-4bf5-a5d5-2ff4823201da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:12.292743 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d9b99-e202-4bf5-a5d5-2ff4823201da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:12.292743 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d9b99-e202-4bf5-a5d5-2ff4823201da admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:12.294411 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d9b99-e202-4bf5-a5d5-2ff4823201da admin admin] PATH_INFO: `/v3/projects/217ea8f8d30b424ea7d21fdbf40072bd` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:12.302329 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d9b99-e202-4bf5-a5d5-2ff4823201da admin admin] RBAC: Authorizing `identity:delete_project(project_id=217ea8f8d30b424ea7d21fdbf40072bd)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:12.305011 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d9b99-e202-4bf5-a5d5-2ff4823201da admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:12.402931 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 712/1437] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:12.414670 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1e663-e1b2-4bb1-82a6-7f1ef0ae5152 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:12.423153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-45b1e663-e1b2-4bb1-82a6-7f1ef0ae5152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:12.541880 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-7b1d9b99-e202-4bf5-a5d5-2ff4823201da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '217ea8f8d30b424ea7d21fdbf40072bd', 'request_id': 'req-7b1d9b99-e202-4bf5-a5d5-2ff4823201da'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:12.559811 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-7b1d9b99-e202-4bf5-a5d5-2ff4823201da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '217ea8f8d30b424ea7d21fdbf40072bd', 'request_id': 'req-7b1d9b99-e202-4bf5-a5d5-2ff4823201da'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:12.599083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-7b1d9b99-e202-4bf5-a5d5-2ff4823201da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '217ea8f8d30b424ea7d21fdbf40072bd', 'request_id': 'req-7b1d9b99-e202-4bf5-a5d5-2ff4823201da'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:12.624617 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-7b1d9b99-e202-4bf5-a5d5-2ff4823201da admin admin] The token cache is being invalidate because project 217ea8f8d30b424ea7d21fdbf40072bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:12.626698 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-7b1d9b99-e202-4bf5-a5d5-2ff4823201da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:12.628894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-7b1d9b99-e202-4bf5-a5d5-2ff4823201da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:12.634680 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 726/1438] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:12 2022] DELETE /identity/v3/projects/217ea8f8d30b424ea7d21fdbf40072bd => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:12.641095 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d9b99-e202-4bf5-a5d5-2ff4823201da admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:12.642883 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-7b1d9b99-e202-4bf5-a5d5-2ff4823201da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:12.683248 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:12.706439 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1e663-e1b2-4bb1-82a6-7f1ef0ae5152 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:12.858948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d9b99-e202-4bf5-a5d5-2ff4823201da admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:12.887547 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850e2a6-4cfa-4f18-b775-bc2ed8d53669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:12.888225 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850e2a6-4cfa-4f18-b775-bc2ed8d53669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:12.888529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850e2a6-4cfa-4f18-b775-bc2ed8d53669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:12.888818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850e2a6-4cfa-4f18-b775-bc2ed8d53669 admin admin] PATH_INFO: `/v3/users/101926168e2940868d275895a3bf5eee` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:12.897532 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850e2a6-4cfa-4f18-b775-bc2ed8d53669 admin admin] RBAC: Authorizing `identity:delete_user(user_id=101926168e2940868d275895a3bf5eee)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:12.899506 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850e2a6-4cfa-4f18-b775-bc2ed8d53669 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:12.910779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184f27c2-e9ad-4fec-8378-db1186fe4b36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:12.911417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f27c2-e9ad-4fec-8378-db1186fe4b36 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:12.911701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f27c2-e9ad-4fec-8378-db1186fe4b36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:12.911985 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184f27c2-e9ad-4fec-8378-db1186fe4b36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:12.917452 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-184f27c2-e9ad-4fec-8378-db1186fe4b36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:12.962746 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f27c2-e9ad-4fec-8378-db1186fe4b36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:12.973739 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184f27c2-e9ad-4fec-8378-db1186fe4b36 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:12.989258 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:13.017012 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 713/1439] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:08:13.076932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4850e2a6-4cfa-4f18-b775-bc2ed8d53669 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '101926168e2940868d275895a3bf5eee'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:13.107687 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4850e2a6-4cfa-4f18-b775-bc2ed8d53669 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '101926168e2940868d275895a3bf5eee'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:13.134450 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19f020-7821-42c3-8c94-5f83cd13a6f5 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:13.134744 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19f020-7821-42c3-8c94-5f83cd13a6f5 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:13.135024 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19f020-7821-42c3-8c94-5f83cd13a6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:13.172131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4850e2a6-4cfa-4f18-b775-bc2ed8d53669 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '101926168e2940868d275895a3bf5eee'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:13.176047 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4850e2a6-4cfa-4f18-b775-bc2ed8d53669 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '101926168e2940868d275895a3bf5eee'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:13.178817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4850e2a6-4cfa-4f18-b775-bc2ed8d53669 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '101926168e2940868d275895a3bf5eee'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:13.221271 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 727/1440] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:12 2022] DELETE /identity/v3/users/101926168e2940868d275895a3bf5eee => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:13.337478 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-7e19f020-7821-42c3-8c94-5f83cd13a6f5 None None] MFA Rules not processed for user `7c0356072f92451f98b03a2225e22f86`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:13.338728 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-7e19f020-7821-42c3-8c94-5f83cd13a6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:13.506582 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 714/1441] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 00:08:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:13.708141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850e2a6-4cfa-4f18-b775-bc2ed8d53669 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:13.710681 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-4850e2a6-4cfa-4f18-b775-bc2ed8d53669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:13.873772 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19f020-7821-42c3-8c94-5f83cd13a6f5 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:13.875754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-7e19f020-7821-42c3-8c94-5f83cd13a6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:13.977427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850e2a6-4cfa-4f18-b775-bc2ed8d53669 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:13.997803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608eda09-7ea5-43d4-843d-6925ff5c120d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfd8c7516b7c435d88298672b2596d6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:13.998465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608eda09-7ea5-43d4-843d-6925ff5c120d service swift] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:14.003177 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608eda09-7ea5-43d4-843d-6925ff5c120d service swift] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:14.003492 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608eda09-7ea5-43d4-843d-6925ff5c120d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:14.026935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19f020-7821-42c3-8c94-5f83cd13a6f5 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:14.040271 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c7ce5-d9ae-4cf2-93a8-ea8786d68ae6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:14.041066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c7ce5-d9ae-4cf2-93a8-ea8786d68ae6 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:14.041353 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c7ce5-d9ae-4cf2-93a8-ea8786d68ae6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:14.041637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c7ce5-d9ae-4cf2-93a8-ea8786d68ae6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:14.045429 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608eda09-7ea5-43d4-843d-6925ff5c120d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:14.047391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608eda09-7ea5-43d4-843d-6925ff5c120d service swift] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:14.048408 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b98c7ce5-d9ae-4cf2-93a8-ea8786d68ae6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:14.074159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 728/1442] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 00:08:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:08:14.074557 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608eda09-7ea5-43d4-843d-6925ff5c120d service swift] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:14.077818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-608eda09-7ea5-43d4-843d-6925ff5c120d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:14.154680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:14.257730 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c7ce5-d9ae-4cf2-93a8-ea8786d68ae6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:14.260138 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c7ce5-d9ae-4cf2-93a8-ea8786d68ae6 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:14.288834 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 715/1443] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:13 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:08:14.317507 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608eda09-7ea5-43d4-843d-6925ff5c120d service swift] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:14.324175 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dce6df0-84b1-4d79-ae75-15c22003a5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:14.325727 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce6df0-84b1-4d79-ae75-15c22003a5b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:14.326026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce6df0-84b1-4d79-ae75-15c22003a5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:14.326957 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce6df0-84b1-4d79-ae75-15c22003a5b3 admin admin] PATH_INFO: `/v3/projects/6b3e157518b641c1a4f361add48bca80` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:14.330839 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dce6df0-84b1-4d79-ae75-15c22003a5b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b3e157518b641c1a4f361add48bca80)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:14.334989 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dce6df0-84b1-4d79-ae75-15c22003a5b3 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:14.357471 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:14.505792 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-3dce6df0-84b1-4d79-ae75-15c22003a5b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b3e157518b641c1a4f361add48bca80', 'request_id': 'req-3dce6df0-84b1-4d79-ae75-15c22003a5b3'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:14.533235 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-3dce6df0-84b1-4d79-ae75-15c22003a5b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b3e157518b641c1a4f361add48bca80', 'request_id': 'req-3dce6df0-84b1-4d79-ae75-15c22003a5b3'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:14.550887 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-3dce6df0-84b1-4d79-ae75-15c22003a5b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b3e157518b641c1a4f361add48bca80', 'request_id': 'req-3dce6df0-84b1-4d79-ae75-15c22003a5b3'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:14.584554 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-3dce6df0-84b1-4d79-ae75-15c22003a5b3 admin admin] The token cache is being invalidate because project 6b3e157518b641c1a4f361add48bca80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:14.584862 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-3dce6df0-84b1-4d79-ae75-15c22003a5b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:14.585733 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-3dce6df0-84b1-4d79-ae75-15c22003a5b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:14.587639 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 729/1444] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:14 2022] DELETE /identity/v3/projects/6b3e157518b641c1a4f361add48bca80 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:14.591885 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c7ce5-d9ae-4cf2-93a8-ea8786d68ae6 service cinder] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:14.616664 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c7ce5-d9ae-4cf2-93a8-ea8786d68ae6 service cinder] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:14.864844 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caacda5a-9557-4734-b28e-640730aaa85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:14.870661 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacda5a-9557-4734-b28e-640730aaa85b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:14.875971 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacda5a-9557-4734-b28e-640730aaa85b admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:14.879753 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacda5a-9557-4734-b28e-640730aaa85b admin admin] PATH_INFO: `/v3/users/713ac404acf343888cf2d44ad16748bc` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:14.888173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caacda5a-9557-4734-b28e-640730aaa85b admin admin] RBAC: Authorizing `identity:delete_user(user_id=713ac404acf343888cf2d44ad16748bc)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:14.899980 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caacda5a-9557-4734-b28e-640730aaa85b admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:15.088785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-caacda5a-9557-4734-b28e-640730aaa85b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '713ac404acf343888cf2d44ad16748bc'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:15.097717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-caacda5a-9557-4734-b28e-640730aaa85b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '713ac404acf343888cf2d44ad16748bc'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:15.125344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-caacda5a-9557-4734-b28e-640730aaa85b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '713ac404acf343888cf2d44ad16748bc'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:15.184963 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-caacda5a-9557-4734-b28e-640730aaa85b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '713ac404acf343888cf2d44ad16748bc'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:15.214491 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-caacda5a-9557-4734-b28e-640730aaa85b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '713ac404acf343888cf2d44ad16748bc'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:15.302079 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 716/1445] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:14 2022] DELETE /identity/v3/users/713ac404acf343888cf2d44ad16748bc => generated 0 bytes in 711 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:16.293678 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dce6df0-84b1-4d79-ae75-15c22003a5b3 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:16.295277 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-3dce6df0-84b1-4d79-ae75-15c22003a5b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:16.320268 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dce6df0-84b1-4d79-ae75-15c22003a5b3 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:16.510550 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac3862-bd6e-4434-ba14-ccdc3bb50a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:16.511530 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac3862-bd6e-4434-ba14-ccdc3bb50a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:16.512028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac3862-bd6e-4434-ba14-ccdc3bb50a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:16.512570 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac3862-bd6e-4434-ba14-ccdc3bb50a0b admin admin] PATH_INFO: `/v3/projects/3c27369a433e4d9bb209c24a6980ca1c` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:16.516431 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac3862-bd6e-4434-ba14-ccdc3bb50a0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c27369a433e4d9bb209c24a6980ca1c)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:16.526387 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac3862-bd6e-4434-ba14-ccdc3bb50a0b admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:16.579568 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693beb28-9813-4c1a-a2f6-0d1c37a07855 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:16.580825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693beb28-9813-4c1a-a2f6-0d1c37a07855 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:16.581324 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693beb28-9813-4c1a-a2f6-0d1c37a07855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:16.652724 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-693beb28-9813-4c1a-a2f6-0d1c37a07855 None None] MFA Rules not processed for user `a09d45d080bc461c88f38ff281b75b06`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:16.656418 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-693beb28-9813-4c1a-a2f6-0d1c37a07855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:16.750334 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1cac3862-bd6e-4434-ba14-ccdc3bb50a0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c27369a433e4d9bb209c24a6980ca1c', 'request_id': 'req-1cac3862-bd6e-4434-ba14-ccdc3bb50a0b'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:16.768581 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1cac3862-bd6e-4434-ba14-ccdc3bb50a0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c27369a433e4d9bb209c24a6980ca1c', 'request_id': 'req-1cac3862-bd6e-4434-ba14-ccdc3bb50a0b'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:16.786181 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:16.786912 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1cac3862-bd6e-4434-ba14-ccdc3bb50a0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c27369a433e4d9bb209c24a6980ca1c', 'request_id': 'req-1cac3862-bd6e-4434-ba14-ccdc3bb50a0b'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:16.813653 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 717/1446] 10.222.0.15 () {58 vars in 1015 bytes} [Sun Oct 23 00:08:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:08:16.813800 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1cac3862-bd6e-4434-ba14-ccdc3bb50a0b admin admin] The token cache is being invalidate because project 3c27369a433e4d9bb209c24a6980ca1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:16.815013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1cac3862-bd6e-4434-ba14-ccdc3bb50a0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:16.820105 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1cac3862-bd6e-4434-ba14-ccdc3bb50a0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:16.824146 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 730/1447] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:16 2022] DELETE /identity/v3/projects/3c27369a433e4d9bb209c24a6980ca1c => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:16.829714 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8118c-76ac-4107-a655-f63f1a91e10b None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:16.830203 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8118c-76ac-4107-a655-f63f1a91e10b None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:16.830649 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8118c-76ac-4107-a655-f63f1a91e10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:16.870327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-dea8118c-76ac-4107-a655-f63f1a91e10b None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:16.870536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac3862-bd6e-4434-ba14-ccdc3bb50a0b admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:16.871893 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-dea8118c-76ac-4107-a655-f63f1a91e10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:16.872919 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1cac3862-bd6e-4434-ba14-ccdc3bb50a0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:16.946742 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:17.029427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 718/1448] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:17.032756 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac3862-bd6e-4434-ba14-ccdc3bb50a0b admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:17.036778 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5cc56-6c65-42d1-8994-06b6287550da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:17.037410 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5cc56-6c65-42d1-8994-06b6287550da service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:17.037697 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5cc56-6c65-42d1-8994-06b6287550da service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:17.037977 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5cc56-6c65-42d1-8994-06b6287550da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:17.038678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ac253-f06f-4ee4-8ab4-486d7dbea18d None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:17.039290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ac253-f06f-4ee4-8ab4-486d7dbea18d None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:17.039877 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-cdf5cc56-6c65-42d1-8994-06b6287550da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:17.040933 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ac253-f06f-4ee4-8ab4-486d7dbea18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:17.058758 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5cc56-6c65-42d1-8994-06b6287550da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:17.071965 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5cc56-6c65-42d1-8994-06b6287550da service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:17.087269 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-dc4ac253-f06f-4ee4-8ab4-486d7dbea18d None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:17.088273 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 731/1449] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:08:17.129472 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-dc4ac253-f06f-4ee4-8ab4-486d7dbea18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:17.169933 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 719/1450] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:08:17.170257 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5cc56-6c65-42d1-8994-06b6287550da service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:17.186083 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ac253-f06f-4ee4-8ab4-486d7dbea18d None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:17.188634 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-dc4ac253-f06f-4ee4-8ab4-486d7dbea18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:17.203311 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5cc56-6c65-42d1-8994-06b6287550da service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:17.212991 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ac253-f06f-4ee4-8ab4-486d7dbea18d None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:17.219725 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a020f6-3a11-49df-9649-2b6aaf0a46c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:17.220772 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a020f6-3a11-49df-9649-2b6aaf0a46c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:17.221271 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a020f6-3a11-49df-9649-2b6aaf0a46c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:17.221846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a020f6-3a11-49df-9649-2b6aaf0a46c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:17.222978 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a020f6-3a11-49df-9649-2b6aaf0a46c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:08:17.223658 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a020f6-3a11-49df-9649-2b6aaf0a46c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:17.226091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a020f6-3a11-49df-9649-2b6aaf0a46c0 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:17.231784 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b5453-59ab-4d15-9798-17c82baff9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:17.233210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b5453-59ab-4d15-9798-17c82baff9fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:17.234216 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b5453-59ab-4d15-9798-17c82baff9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:17.234472 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b5453-59ab-4d15-9798-17c82baff9fd admin admin] PATH_INFO: `/v3/users/d029a0e121574898b10942d80212db3b` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:17.235925 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b5453-59ab-4d15-9798-17c82baff9fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d029a0e121574898b10942d80212db3b)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:17.238244 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184b5453-59ab-4d15-9798-17c82baff9fd admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:17.250606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 732/1451] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:08:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:08:17.252509 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a020f6-3a11-49df-9649-2b6aaf0a46c0 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:17.255161 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-29a020f6-3a11-49df-9649-2b6aaf0a46c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:17.279834 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a020f6-3a11-49df-9649-2b6aaf0a46c0 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:17.290461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e9f49-b47a-4a38-b453-4b11d93053e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:17.292190 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e9f49-b47a-4a38-b453-4b11d93053e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:17.293340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e9f49-b47a-4a38-b453-4b11d93053e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:17.294149 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e9f49-b47a-4a38-b453-4b11d93053e8 admin admin] PATH_INFO: `/v3/projects/6d730bc61aaa4ef88dc3961fe3f498e8` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:17.300803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e9f49-b47a-4a38-b453-4b11d93053e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d730bc61aaa4ef88dc3961fe3f498e8)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:17.306609 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e9f49-b47a-4a38-b453-4b11d93053e8 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:17.345745 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-184b5453-59ab-4d15-9798-17c82baff9fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd029a0e121574898b10942d80212db3b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:17.346997 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-184b5453-59ab-4d15-9798-17c82baff9fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd029a0e121574898b10942d80212db3b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:17.372552 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-184b5453-59ab-4d15-9798-17c82baff9fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd029a0e121574898b10942d80212db3b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:17.411600 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-184b5453-59ab-4d15-9798-17c82baff9fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd029a0e121574898b10942d80212db3b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:17.421445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-de8e9f49-b47a-4a38-b453-4b11d93053e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d730bc61aaa4ef88dc3961fe3f498e8', 'request_id': 'req-de8e9f49-b47a-4a38-b453-4b11d93053e8'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:17.430674 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-184b5453-59ab-4d15-9798-17c82baff9fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd029a0e121574898b10942d80212db3b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:17.441695 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 720/1452] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:17 2022] DELETE /identity/v3/users/d029a0e121574898b10942d80212db3b => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:08:17.441905 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-de8e9f49-b47a-4a38-b453-4b11d93053e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d730bc61aaa4ef88dc3961fe3f498e8', 'request_id': 'req-de8e9f49-b47a-4a38-b453-4b11d93053e8'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:17.442552 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b5453-59ab-4d15-9798-17c82baff9fd admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:17.445575 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-184b5453-59ab-4d15-9798-17c82baff9fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:17.457097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-de8e9f49-b47a-4a38-b453-4b11d93053e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d730bc61aaa4ef88dc3961fe3f498e8', 'request_id': 'req-de8e9f49-b47a-4a38-b453-4b11d93053e8'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:17.484159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-de8e9f49-b47a-4a38-b453-4b11d93053e8 admin admin] The token cache is being invalidate because project 6d730bc61aaa4ef88dc3961fe3f498e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:17.484436 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-de8e9f49-b47a-4a38-b453-4b11d93053e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:17.485198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-de8e9f49-b47a-4a38-b453-4b11d93053e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:17.486898 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 733/1453] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:17 2022] DELETE /identity/v3/projects/6d730bc61aaa4ef88dc3961fe3f498e8 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:17.488344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bc282-b30a-4541-b077-aecd5fdf4a6a None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:17.488798 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bc282-b30a-4541-b077-aecd5fdf4a6a None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:17.489368 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bc282-b30a-4541-b077-aecd5fdf4a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:17.517967 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-746bc282-b30a-4541-b077-aecd5fdf4a6a None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:17.519534 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-746bc282-b30a-4541-b077-aecd5fdf4a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:17.614408 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b5453-59ab-4d15-9798-17c82baff9fd admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:17.626711 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3bf8d2-213a-42f8-a894-0eb935b1fcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:17.627349 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3bf8d2-213a-42f8-a894-0eb935b1fcd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:17.627813 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3bf8d2-213a-42f8-a894-0eb935b1fcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:17.628261 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3bf8d2-213a-42f8-a894-0eb935b1fcd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:17.629136 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3bf8d2-213a-42f8-a894-0eb935b1fcd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:17.631219 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3bf8d2-213a-42f8-a894-0eb935b1fcd4 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:17.659885 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:17.696726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 721/1454] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:17 2022] POST /identity/v3/projects => generated 369 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:17.697994 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3bf8d2-213a-42f8-a894-0eb935b1fcd4 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:17.702705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ec3bf8d2-213a-42f8-a894-0eb935b1fcd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:17.724715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 734/1455] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:17.724715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86a206-cb02-4a09-b530-fe41b2b157d0 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:17.724715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86a206-cb02-4a09-b530-fe41b2b157d0 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:17.724715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86a206-cb02-4a09-b530-fe41b2b157d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:17.754148 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-4b86a206-cb02-4a09-b530-fe41b2b157d0 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:17.755526 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-4b86a206-cb02-4a09-b530-fe41b2b157d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:17.814841 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:17.831718 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3bf8d2-213a-42f8-a894-0eb935b1fcd4 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:17.839259 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736dcd45-d4dd-4345-889f-caf64389b624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:17.840152 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736dcd45-d4dd-4345-889f-caf64389b624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:17.840636 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736dcd45-d4dd-4345-889f-caf64389b624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:17.841081 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736dcd45-d4dd-4345-889f-caf64389b624 admin admin] PATH_INFO: `/v3/projects/f9389ae2621746bcaee099404ca3ce92` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:17.843506 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736dcd45-d4dd-4345-889f-caf64389b624 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9389ae2621746bcaee099404ca3ce92)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:17.845910 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736dcd45-d4dd-4345-889f-caf64389b624 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:17.874436 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 735/1456] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:17.875585 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b86a206-cb02-4a09-b530-fe41b2b157d0 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:17.882777 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-4b86a206-cb02-4a09-b530-fe41b2b157d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:17.913324 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b86a206-cb02-4a09-b530-fe41b2b157d0 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:17.926657 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4104d51-190e-4934-b11e-55267ebdbc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:17.927475 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4104d51-190e-4934-b11e-55267ebdbc3b admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:17.928253 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4104d51-190e-4934-b11e-55267ebdbc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:17.928829 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4104d51-190e-4934-b11e-55267ebdbc3b admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:17.929786 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4104d51-190e-4934-b11e-55267ebdbc3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:17.931882 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4104d51-190e-4934-b11e-55267ebdbc3b admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:17.982957 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 736/1457] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:17 2022] POST /identity/v3/users => generated 285 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:17.984588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4104d51-190e-4934-b11e-55267ebdbc3b admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:18.005550 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a4104d51-190e-4934-b11e-55267ebdbc3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:18.037071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-736dcd45-d4dd-4345-889f-caf64389b624 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9389ae2621746bcaee099404ca3ce92', 'request_id': 'req-736dcd45-d4dd-4345-889f-caf64389b624'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:18.073073 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-736dcd45-d4dd-4345-889f-caf64389b624 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9389ae2621746bcaee099404ca3ce92', 'request_id': 'req-736dcd45-d4dd-4345-889f-caf64389b624'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:18.107810 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-736dcd45-d4dd-4345-889f-caf64389b624 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9389ae2621746bcaee099404ca3ce92', 'request_id': 'req-736dcd45-d4dd-4345-889f-caf64389b624'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:18.144734 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4104d51-190e-4934-b11e-55267ebdbc3b admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:18.164464 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-736dcd45-d4dd-4345-889f-caf64389b624 admin admin] The token cache is being invalidate because project f9389ae2621746bcaee099404ca3ce92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:18.164784 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-736dcd45-d4dd-4345-889f-caf64389b624 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:18.165807 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-736dcd45-d4dd-4345-889f-caf64389b624 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:18.173798 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 722/1458] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:17 2022] DELETE /identity/v3/projects/f9389ae2621746bcaee099404ca3ce92 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:18.175243 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8ed63-161c-4743-92e8-ca450e633912 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:18.175534 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8ed63-161c-4743-92e8-ca450e633912 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:18.175808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b8ed63-161c-4743-92e8-ca450e633912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:18.260256 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-17b8ed63-161c-4743-92e8-ca450e633912 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:18.261554 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-17b8ed63-161c-4743-92e8-ca450e633912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:18.320266 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917c7ec-9335-4d51-9bdc-57b720c0bf03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:18.323280 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917c7ec-9335-4d51-9bdc-57b720c0bf03 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:18.323787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917c7ec-9335-4d51-9bdc-57b720c0bf03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:18.324072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917c7ec-9335-4d51-9bdc-57b720c0bf03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:18.327756 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-0917c7ec-9335-4d51-9bdc-57b720c0bf03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:18.437965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 723/1459] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:18.437965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8ed63-161c-4743-92e8-ca450e633912 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:18.441914 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-17b8ed63-161c-4743-92e8-ca450e633912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:18.468509 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b8ed63-161c-4743-92e8-ca450e633912 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:18.477666 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917c7ec-9335-4d51-9bdc-57b720c0bf03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:18.479955 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917c7ec-9335-4d51-9bdc-57b720c0bf03 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:18.487262 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b7073e-36f3-4c02-b883-42c6686a9610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:18.487883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7073e-36f3-4c02-b883-42c6686a9610 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:18.488163 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7073e-36f3-4c02-b883-42c6686a9610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:18.488434 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b7073e-36f3-4c02-b883-42c6686a9610 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:18.489250 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b7073e-36f3-4c02-b883-42c6686a9610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:18.491057 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b7073e-36f3-4c02-b883-42c6686a9610 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:18.507379 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 737/1460] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:18.511286 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82666b-bc90-45fc-8e9b-de36dee8ddfe None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:18.511517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82666b-bc90-45fc-8e9b-de36dee8ddfe None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:18.511853 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82666b-bc90-45fc-8e9b-de36dee8ddfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:18.513792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 724/1461] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:18.515345 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b7073e-36f3-4c02-b883-42c6686a9610 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:18.534417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b7073e-36f3-4c02-b883-42c6686a9610 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:18.547778 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d468c2-ca6e-47ac-9a24-b719363d5ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:18.549643 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d468c2-ca6e-47ac-9a24-b719363d5ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:18.549883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d468c2-ca6e-47ac-9a24-b719363d5ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:18.551064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d468c2-ca6e-47ac-9a24-b719363d5ca4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:18.552952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d468c2-ca6e-47ac-9a24-b719363d5ca4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:08:18.553257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d468c2-ca6e-47ac-9a24-b719363d5ca4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:18.554872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d468c2-ca6e-47ac-9a24-b719363d5ca4 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:18.557416 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-7d82666b-bc90-45fc-8e9b-de36dee8ddfe None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:18.558635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-7d82666b-bc90-45fc-8e9b-de36dee8ddfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:18.562656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:18.575809 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 738/1462] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:18.584343 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82666b-bc90-45fc-8e9b-de36dee8ddfe None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:18.595118 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 725/1463] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:08:18.595118 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125d118-8926-437e-8cca-8fe35094429b None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:18.595118 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125d118-8926-437e-8cca-8fe35094429b None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:18.595118 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9125d118-8926-437e-8cca-8fe35094429b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:18.603725 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82666b-bc90-45fc-8e9b-de36dee8ddfe None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:18.621592 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec07fc-974a-422a-9988-6df75268218b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:18.622221 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec07fc-974a-422a-9988-6df75268218b admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:18.622502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec07fc-974a-422a-9988-6df75268218b admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:18.623385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec07fc-974a-422a-9988-6df75268218b admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:18.623698 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec07fc-974a-422a-9988-6df75268218b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:18.627505 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec07fc-974a-422a-9988-6df75268218b admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:18.640666 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-9125d118-8926-437e-8cca-8fe35094429b None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:18.641973 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9125d118-8926-437e-8cca-8fe35094429b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:18.650324 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 739/1464] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:18.651183 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec07fc-974a-422a-9988-6df75268218b admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:18.663944 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 726/1465] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:18.666330 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125d118-8926-437e-8cca-8fe35094429b None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:18.696198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec07fc-974a-422a-9988-6df75268218b admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:18.716431 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9125d118-8926-437e-8cca-8fe35094429b None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:18.719741 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c1e9a-925b-4312-b3a5-3bbd64cedfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:18.720868 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c1e9a-925b-4312-b3a5-3bbd64cedfa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:18.721801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c1e9a-925b-4312-b3a5-3bbd64cedfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:18.722362 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c1e9a-925b-4312-b3a5-3bbd64cedfa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:18.723249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c1e9a-925b-4312-b3a5-3bbd64cedfa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:18.726198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c1e9a-925b-4312-b3a5-3bbd64cedfa5 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:18.744149 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba46c5-b716-4c49-a5a1-08760a7b0037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:18.744880 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba46c5-b716-4c49-a5a1-08760a7b0037 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:18.745961 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba46c5-b716-4c49-a5a1-08760a7b0037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:18.746252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba46c5-b716-4c49-a5a1-08760a7b0037 admin admin] PATH_INFO: `/v3/projects/a0a5ea0c39c24ce4acdf4a4626999a57/users/82555cf67aa84244b20511661549700f/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:18.786634 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:18.817666 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba46c5-b716-4c49-a5a1-08760a7b0037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a5ea0c39c24ce4acdf4a4626999a57, user_id=82555cf67aa84244b20511661549700f, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:18.820656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba46c5-b716-4c49-a5a1-08760a7b0037 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:18.852540 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 740/1466] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:18 2022] POST /identity/v3/projects => generated 373 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:18.853676 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c1e9a-925b-4312-b3a5-3bbd64cedfa5 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:18.878764 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 727/1467] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:18 2022] PUT /identity/v3/projects/a0a5ea0c39c24ce4acdf4a4626999a57/users/82555cf67aa84244b20511661549700f/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:18.879590 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba46c5-b716-4c49-a5a1-08760a7b0037 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:18.996865 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c1e9a-925b-4312-b3a5-3bbd64cedfa5 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:19.024223 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba46c5-b716-4c49-a5a1-08760a7b0037 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:19.045815 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ff3e1-fca8-4574-ab47-b8d8c6f2c52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:19.049291 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ff3e1-fca8-4574-ab47-b8d8c6f2c52b admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:19.049788 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ff3e1-fca8-4574-ab47-b8d8c6f2c52b admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:19.051399 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ff3e1-fca8-4574-ab47-b8d8c6f2c52b admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:19.054518 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322ff3e1-fca8-4574-ab47-b8d8c6f2c52b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:19.056709 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322ff3e1-fca8-4574-ab47-b8d8c6f2c52b admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:19.125605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 728/1468] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:18 2022] POST /identity/v3/users => generated 287 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:19.127353 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6874420-479d-4185-a82f-f109e12a2f4e None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:19.127812 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6874420-479d-4185-a82f-f109e12a2f4e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:19.128283 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6874420-479d-4185-a82f-f109e12a2f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:19.191702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e137769-5790-4c9f-b6b7-d0a2b26ee171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:19.195062 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e137769-5790-4c9f-b6b7-d0a2b26ee171 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:19.195973 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-d6874420-479d-4185-a82f-f109e12a2f4e None None] MFA Rules not processed for user `82555cf67aa84244b20511661549700f`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:19.196411 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e137769-5790-4c9f-b6b7-d0a2b26ee171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:19.197542 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d6874420-479d-4185-a82f-f109e12a2f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:19.198489 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e137769-5790-4c9f-b6b7-d0a2b26ee171 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:19.200155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e137769-5790-4c9f-b6b7-d0a2b26ee171 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:08:19.203519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e137769-5790-4c9f-b6b7-d0a2b26ee171 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:19.210345 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e137769-5790-4c9f-b6b7-d0a2b26ee171 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:19.267384 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 741/1469] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:08:19.267384 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e137769-5790-4c9f-b6b7-d0a2b26ee171 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:19.290461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e137769-5790-4c9f-b6b7-d0a2b26ee171 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:19.304412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49da1a4-fec7-4722-9a75-06d1b2f29af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:19.304840 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49da1a4-fec7-4722-9a75-06d1b2f29af3 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:19.305250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49da1a4-fec7-4722-9a75-06d1b2f29af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:19.305751 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49da1a4-fec7-4722-9a75-06d1b2f29af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:19.306552 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49da1a4-fec7-4722-9a75-06d1b2f29af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:19.308294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49da1a4-fec7-4722-9a75-06d1b2f29af3 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:19.337013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 742/1470] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:19.339584 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49da1a4-fec7-4722-9a75-06d1b2f29af3 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:19.362840 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49da1a4-fec7-4722-9a75-06d1b2f29af3 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:19.369492 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 729/1471] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:19.370763 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6874420-479d-4185-a82f-f109e12a2f4e None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:19.385225 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6daab-ac66-441c-a8bc-8fe44b50f664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:19.388083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6daab-ac66-441c-a8bc-8fe44b50f664 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:19.388526 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6daab-ac66-441c-a8bc-8fe44b50f664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:19.389351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6daab-ac66-441c-a8bc-8fe44b50f664 admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:19.390740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6daab-ac66-441c-a8bc-8fe44b50f664 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:19.397045 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6874420-479d-4185-a82f-f109e12a2f4e None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:19.398738 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6daab-ac66-441c-a8bc-8fe44b50f664 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:19.409282 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf243c-57c1-440e-8967-d6ee72cec49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:19.410004 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf243c-57c1-440e-8967-d6ee72cec49b admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:19.410215 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf243c-57c1-440e-8967-d6ee72cec49b admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:19.410515 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf243c-57c1-440e-8967-d6ee72cec49b admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:19.411192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf243c-57c1-440e-8967-d6ee72cec49b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:19.413087 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf243c-57c1-440e-8967-d6ee72cec49b admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:19.448319 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 730/1472] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:19.454429 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf243c-57c1-440e-8967-d6ee72cec49b admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:19.488683 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf243c-57c1-440e-8967-d6ee72cec49b admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:19.499910 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 743/1473] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:19 2022] POST /identity/v3/projects => generated 363 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:19.499910 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6daab-ac66-441c-a8bc-8fe44b50f664 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:19.508115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924b024b-6da8-46f6-ba93-a2b12e54d044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:19.508720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b024b-6da8-46f6-ba93-a2b12e54d044 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:19.509010 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b024b-6da8-46f6-ba93-a2b12e54d044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:19.509401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b024b-6da8-46f6-ba93-a2b12e54d044 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:19.510058 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924b024b-6da8-46f6-ba93-a2b12e54d044 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:19.512116 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924b024b-6da8-46f6-ba93-a2b12e54d044 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:19.609910 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 731/1474] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:19 2022] POST /identity/v3/projects => generated 367 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:19.617899 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924b024b-6da8-46f6-ba93-a2b12e54d044 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:19.651958 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:19.674864 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:19.679552 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6daab-ac66-441c-a8bc-8fe44b50f664 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:19.812697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924b024b-6da8-46f6-ba93-a2b12e54d044 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:19.833851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d070b-f224-4d88-a2be-38982a484eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:19.834994 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d070b-f224-4d88-a2be-38982a484eaa admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:19.835326 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d070b-f224-4d88-a2be-38982a484eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:19.838470 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d070b-f224-4d88-a2be-38982a484eaa admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:19.839371 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2d070b-f224-4d88-a2be-38982a484eaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:19.843191 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2d070b-f224-4d88-a2be-38982a484eaa admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:19.867929 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fab7e0-ffb7-4cd2-b02a-87ad8a78a107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:19.871238 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fab7e0-ffb7-4cd2-b02a-87ad8a78a107 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:19.871535 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fab7e0-ffb7-4cd2-b02a-87ad8a78a107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:19.871824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fab7e0-ffb7-4cd2-b02a-87ad8a78a107 admin admin] PATH_INFO: `/v3/projects/08d343eb97fc46e29e2ba18751273dbf/users/044624f790b243ee895ae65c4fa729eb/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:19.910350 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 732/1475] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:19 2022] POST /identity/v3/users => generated 282 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:19.919468 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d070b-f224-4d88-a2be-38982a484eaa admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:19.948529 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d070b-f224-4d88-a2be-38982a484eaa admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:19.952271 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fab7e0-ffb7-4cd2-b02a-87ad8a78a107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08d343eb97fc46e29e2ba18751273dbf, user_id=044624f790b243ee895ae65c4fa729eb, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:19.955668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fab7e0-ffb7-4cd2-b02a-87ad8a78a107 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:19.964075 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48820048-89b5-4a70-894b-a4b28d62d3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:19.964957 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48820048-89b5-4a70-894b-a4b28d62d3a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:19.965528 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48820048-89b5-4a70-894b-a4b28d62d3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:19.970942 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48820048-89b5-4a70-894b-a4b28d62d3a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:19.972023 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48820048-89b5-4a70-894b-a4b28d62d3a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:19.979664 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48820048-89b5-4a70-894b-a4b28d62d3a7 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:20.012399 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 744/1476] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:19 2022] PUT /identity/v3/projects/08d343eb97fc46e29e2ba18751273dbf/users/044624f790b243ee895ae65c4fa729eb/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:20.013202 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fab7e0-ffb7-4cd2-b02a-87ad8a78a107 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:20.036346 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 733/1477] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:19 2022] POST /identity/v3/users => generated 290 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:20.037448 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3cf3e-3ce0-421c-a23e-457bda91a140 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:20.037755 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3cf3e-3ce0-421c-a23e-457bda91a140 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:20.038038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3cf3e-3ce0-421c-a23e-457bda91a140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:20.151393 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-75b3cf3e-3ce0-421c-a23e-457bda91a140 None None] MFA Rules not processed for user `044624f790b243ee895ae65c4fa729eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:20.152997 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-75b3cf3e-3ce0-421c-a23e-457bda91a140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:20.240357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fab7e0-ffb7-4cd2-b02a-87ad8a78a107 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:20.257272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8359187-1712-43d5-b348-58442dd801db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:20.257926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8359187-1712-43d5-b348-58442dd801db admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:20.258256 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8359187-1712-43d5-b348-58442dd801db admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:20.258553 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8359187-1712-43d5-b348-58442dd801db admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:20.259322 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8359187-1712-43d5-b348-58442dd801db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:20.261053 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8359187-1712-43d5-b348-58442dd801db admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:20.302057 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 745/1478] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:20.302962 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8359187-1712-43d5-b348-58442dd801db admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:20.328415 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8359187-1712-43d5-b348-58442dd801db admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:20.346422 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f3707-af7f-4879-bf37-e1d998a52395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:20.346914 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f3707-af7f-4879-bf37-e1d998a52395 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:20.347066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f3707-af7f-4879-bf37-e1d998a52395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:20.347353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f3707-af7f-4879-bf37-e1d998a52395 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:20.348118 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f3707-af7f-4879-bf37-e1d998a52395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:20.350141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f3707-af7f-4879-bf37-e1d998a52395 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:20.350710 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 734/1479] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:20.354214 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3cf3e-3ce0-421c-a23e-457bda91a140 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:20.371898 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 746/1480] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:20.372948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f3707-af7f-4879-bf37-e1d998a52395 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:20.378511 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-690f3707-af7f-4879-bf37-e1d998a52395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:20.390063 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3cf3e-3ce0-421c-a23e-457bda91a140 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:20.409121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4121e3e3-786a-4301-9733-f2ab8f457b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:20.411619 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121e3e3-786a-4301-9733-f2ab8f457b5a admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:20.411915 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121e3e3-786a-4301-9733-f2ab8f457b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:20.413332 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121e3e3-786a-4301-9733-f2ab8f457b5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:20.414791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4121e3e3-786a-4301-9733-f2ab8f457b5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:20.422545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4121e3e3-786a-4301-9733-f2ab8f457b5a admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:20.446788 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 735/1481] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:20.447775 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4121e3e3-786a-4301-9733-f2ab8f457b5a admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:20.449991 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-4121e3e3-786a-4301-9733-f2ab8f457b5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:20.559620 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f3707-af7f-4879-bf37-e1d998a52395 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:20.559826 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4121e3e3-786a-4301-9733-f2ab8f457b5a admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:20.565381 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397ceb4-f342-4129-b5bb-f608ae08e2b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:20.566297 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397ceb4-f342-4129-b5bb-f608ae08e2b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:20.566616 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397ceb4-f342-4129-b5bb-f608ae08e2b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:20.566896 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397ceb4-f342-4129-b5bb-f608ae08e2b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:20.571236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-1397ceb4-f342-4129-b5bb-f608ae08e2b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:20.579128 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d0d7a-95ea-45d9-acf8-6a61ffbf5750 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:20.579910 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d0d7a-95ea-45d9-acf8-6a61ffbf5750 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:20.580280 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d0d7a-95ea-45d9-acf8-6a61ffbf5750 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:20.580587 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3d0d7a-95ea-45d9-acf8-6a61ffbf5750 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:20.591809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:20.602755 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d0d7a-95ea-45d9-acf8-6a61ffbf5750 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:20.604782 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3d0d7a-95ea-45d9-acf8-6a61ffbf5750 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:20.611910 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:20.618370 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 747/1482] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:20.619716 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d0d7a-95ea-45d9-acf8-6a61ffbf5750 service nova] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:20.633931 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3d0d7a-95ea-45d9-acf8-6a61ffbf5750 service nova] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:20.640676 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b48e30-033a-4b8c-be60-c23e6c14c07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:20.641546 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b48e30-033a-4b8c-be60-c23e6c14c07e admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:20.641835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b48e30-033a-4b8c-be60-c23e6c14c07e admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:20.642115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b48e30-033a-4b8c-be60-c23e6c14c07e admin admin] PATH_INFO: `/v3/projects/5031ec41c22d40e680521e0d3621292e/users/9accccf5dcf3431ca82cb13493f9d58c/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:20.698201 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b48e30-033a-4b8c-be60-c23e6c14c07e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5031ec41c22d40e680521e0d3621292e, user_id=9accccf5dcf3431ca82cb13493f9d58c, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:20.707097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b48e30-033a-4b8c-be60-c23e6c14c07e admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:20.748242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1397ceb4-f342-4129-b5bb-f608ae08e2b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:20.753430 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1397ceb4-f342-4129-b5bb-f608ae08e2b2 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:20.754387 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 748/1483] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:20 2022] PUT /identity/v3/projects/5031ec41c22d40e680521e0d3621292e/users/9accccf5dcf3431ca82cb13493f9d58c/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:20.755561 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b48e30-033a-4b8c-be60-c23e6c14c07e admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:20.778628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 736/1484] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:20.779660 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397ceb4-f342-4129-b5bb-f608ae08e2b2 service cinder] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:20.813267 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397ceb4-f342-4129-b5bb-f608ae08e2b2 service cinder] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:20.993281 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61132478-cd0f-4e46-bdb3-f48590117702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:20.993905 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61132478-cd0f-4e46-bdb3-f48590117702 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:20.994186 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61132478-cd0f-4e46-bdb3-f48590117702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:20.994462 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61132478-cd0f-4e46-bdb3-f48590117702 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:20.995170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61132478-cd0f-4e46-bdb3-f48590117702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:21.016651 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61132478-cd0f-4e46-bdb3-f48590117702 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:21.052633 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 737/1485] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:21.054137 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead2858-7357-4d71-8bba-3f66c8ffe977 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:21.062926 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead2858-7357-4d71-8bba-3f66c8ffe977 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:21.062926 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead2858-7357-4d71-8bba-3f66c8ffe977 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:21.069265 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 738/1486] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 00:08:21 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:08:21.070577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead2858-7357-4d71-8bba-3f66c8ffe977 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:21.082204 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b48e30-033a-4b8c-be60-c23e6c14c07e admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:21.105379 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79cf30-551d-45ca-9d06-941e32bc40f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:21.106096 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79cf30-551d-45ca-9d06-941e32bc40f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:21.106380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79cf30-551d-45ca-9d06-941e32bc40f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:21.106668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79cf30-551d-45ca-9d06-941e32bc40f4 admin admin] PATH_INFO: `/v3/projects/eff745c2bd5f4dfa825ef57cf81c3ae2/users/c3823a85a5d54c9e8a4a2556e73a2c81/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:21.132766 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead2858-7357-4d71-8bba-3f66c8ffe977 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:21.146808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23b256-6cce-4043-bd7a-6b7463bff983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:21.149293 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23b256-6cce-4043-bd7a-6b7463bff983 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:21.150248 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23b256-6cce-4043-bd7a-6b7463bff983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:21.151279 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23b256-6cce-4043-bd7a-6b7463bff983 admin admin] PATH_INFO: `/v3/projects/5031ec41c22d40e680521e0d3621292e/users/9accccf5dcf3431ca82cb13493f9d58c/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:21.161138 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab23b256-6cce-4043-bd7a-6b7463bff983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5031ec41c22d40e680521e0d3621292e, user_id=9accccf5dcf3431ca82cb13493f9d58c, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:21.167036 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79cf30-551d-45ca-9d06-941e32bc40f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eff745c2bd5f4dfa825ef57cf81c3ae2, user_id=c3823a85a5d54c9e8a4a2556e73a2c81, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:21.172385 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab23b256-6cce-4043-bd7a-6b7463bff983 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:21.181098 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79cf30-551d-45ca-9d06-941e32bc40f4 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:21.199183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 739/1487] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:21 2022] PUT /identity/v3/projects/5031ec41c22d40e680521e0d3621292e/users/9accccf5dcf3431ca82cb13493f9d58c/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:21.214066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287ec9f-2f9d-4a21-aed7-a032b1342eec None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:21.214347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287ec9f-2f9d-4a21-aed7-a032b1342eec None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:21.214647 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287ec9f-2f9d-4a21-aed7-a032b1342eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:21.255855 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 749/1488] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:20 2022] PUT /identity/v3/projects/eff745c2bd5f4dfa825ef57cf81c3ae2/users/c3823a85a5d54c9e8a4a2556e73a2c81/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:21.266907 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b1915-4c3d-400b-9388-1d0c3b854c13 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:21.271411 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b1915-4c3d-400b-9388-1d0c3b854c13 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:21.271729 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b1915-4c3d-400b-9388-1d0c3b854c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:21.282891 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-c287ec9f-2f9d-4a21-aed7-a032b1342eec None None] MFA Rules not processed for user `9accccf5dcf3431ca82cb13493f9d58c`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:21.284151 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c287ec9f-2f9d-4a21-aed7-a032b1342eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:21.327576 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1e3b1915-4c3d-400b-9388-1d0c3b854c13 None None] MFA Rules not processed for user `c4a3579d56294b9d8ad36d4d174c8526`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:21.328903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1e3b1915-4c3d-400b-9388-1d0c3b854c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:21.440462 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 740/1489] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:21 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:08:21.443961 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de342c-a1f9-487f-bb7e-49c8fd6e5289 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:21.447542 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de342c-a1f9-487f-bb7e-49c8fd6e5289 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:21.448067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de342c-a1f9-487f-bb7e-49c8fd6e5289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:21.458779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 750/1490] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 00:08:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:21.546865 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-82de342c-a1f9-487f-bb7e-49c8fd6e5289 None None] MFA Rules not processed for user `c3823a85a5d54c9e8a4a2556e73a2c81`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:21.549765 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-82de342c-a1f9-487f-bb7e-49c8fd6e5289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:21.594328 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:21.694601 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b1915-4c3d-400b-9388-1d0c3b854c13 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:21.696714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1e3b1915-4c3d-400b-9388-1d0c3b854c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:21.829036 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 741/1491] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:21.833248 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de342c-a1f9-487f-bb7e-49c8fd6e5289 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:21.865594 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:21.878022 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b1915-4c3d-400b-9388-1d0c3b854c13 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:21.887107 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115a0c8-428a-41fc-a4f6-ce3a3cae530c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:21.888365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115a0c8-428a-41fc-a4f6-ce3a3cae530c service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:21.888882 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115a0c8-428a-41fc-a4f6-ce3a3cae530c service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:21.889410 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115a0c8-428a-41fc-a4f6-ce3a3cae530c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:21.902048 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115a0c8-428a-41fc-a4f6-ce3a3cae530c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:21.903934 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115a0c8-428a-41fc-a4f6-ce3a3cae530c service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:21.936948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 751/1492] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:22.038887 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de342c-a1f9-487f-bb7e-49c8fd6e5289 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:22.041524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115a0c8-428a-41fc-a4f6-ce3a3cae530c service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:22.051999 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec961142-8fe8-4399-b84f-ae40d3d0196e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:22.053459 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec961142-8fe8-4399-b84f-ae40d3d0196e admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:22.053987 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec961142-8fe8-4399-b84f-ae40d3d0196e admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:22.054454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec961142-8fe8-4399-b84f-ae40d3d0196e admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:22.055272 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec961142-8fe8-4399-b84f-ae40d3d0196e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:22.055699 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115a0c8-428a-41fc-a4f6-ce3a3cae530c service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:22.061892 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec961142-8fe8-4399-b84f-ae40d3d0196e admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:22.068626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065ea54-8ae2-4783-bfdb-c5160177375c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:22.070369 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065ea54-8ae2-4783-bfdb-c5160177375c service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:22.072731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065ea54-8ae2-4783-bfdb-c5160177375c service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:22.074632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065ea54-8ae2-4783-bfdb-c5160177375c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:22.178066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 742/1493] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:21 2022] POST /identity/v3/projects => generated 363 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:08:22.179901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec961142-8fe8-4399-b84f-ae40d3d0196e admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:22.332566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec961142-8fe8-4399-b84f-ae40d3d0196e admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:22.337826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065ea54-8ae2-4783-bfdb-c5160177375c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:22.339390 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a065ea54-8ae2-4783-bfdb-c5160177375c service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:22.349517 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490269a-a84e-4099-bd59-d54f927e9583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:22.350152 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490269a-a84e-4099-bd59-d54f927e9583 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:22.350953 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 752/1494] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:22.361681 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490269a-a84e-4099-bd59-d54f927e9583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:22.362117 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490269a-a84e-4099-bd59-d54f927e9583 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:22.362810 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490269a-a84e-4099-bd59-d54f927e9583 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:22.364801 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490269a-a84e-4099-bd59-d54f927e9583 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:22.422169 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065ea54-8ae2-4783-bfdb-c5160177375c service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:22.423594 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a065ea54-8ae2-4783-bfdb-c5160177375c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:22.435437 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 743/1495] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:22 2022] POST /identity/v3/users => generated 288 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:22.441266 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490269a-a84e-4099-bd59-d54f927e9583 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:22.454083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065ea54-8ae2-4783-bfdb-c5160177375c service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:22.465290 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5194b-123e-4eeb-98f6-f2f36dd8c4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:22.466167 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5194b-123e-4eeb-98f6-f2f36dd8c4af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:22.466452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5194b-123e-4eeb-98f6-f2f36dd8c4af admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:22.467227 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5194b-123e-4eeb-98f6-f2f36dd8c4af admin admin] PATH_INFO: `/v3/projects/441c14c2698c4439b9d90f7e2ecfc1ad` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:22.472033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490269a-a84e-4099-bd59-d54f927e9583 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:22.473114 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5194b-123e-4eeb-98f6-f2f36dd8c4af admin admin] RBAC: Authorizing `identity:delete_project(project_id=441c14c2698c4439b9d90f7e2ecfc1ad)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:22.475764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5194b-123e-4eeb-98f6-f2f36dd8c4af admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:22.483091 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ef8dc-3f86-459b-aa3a-584cfa6d1fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:22.484583 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ef8dc-3f86-459b-aa3a-584cfa6d1fda admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:22.484849 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ef8dc-3f86-459b-aa3a-584cfa6d1fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:22.486008 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6ef8dc-3f86-459b-aa3a-584cfa6d1fda admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:22.486008 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ef8dc-3f86-459b-aa3a-584cfa6d1fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:22.491925 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6ef8dc-3f86-459b-aa3a-584cfa6d1fda admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:22.528752 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 744/1496] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:22.529770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ef8dc-3f86-459b-aa3a-584cfa6d1fda admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:22.594154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6ef8dc-3f86-459b-aa3a-584cfa6d1fda admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:22.642198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-72d5194b-123e-4eeb-98f6-f2f36dd8c4af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '441c14c2698c4439b9d90f7e2ecfc1ad', 'request_id': 'req-72d5194b-123e-4eeb-98f6-f2f36dd8c4af'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:22.644379 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:22.655028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-72d5194b-123e-4eeb-98f6-f2f36dd8c4af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '441c14c2698c4439b9d90f7e2ecfc1ad', 'request_id': 'req-72d5194b-123e-4eeb-98f6-f2f36dd8c4af'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:22.665939 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-72d5194b-123e-4eeb-98f6-f2f36dd8c4af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '441c14c2698c4439b9d90f7e2ecfc1ad', 'request_id': 'req-72d5194b-123e-4eeb-98f6-f2f36dd8c4af'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:22.695147 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-72d5194b-123e-4eeb-98f6-f2f36dd8c4af admin admin] The token cache is being invalidate because project 441c14c2698c4439b9d90f7e2ecfc1ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:22.695345 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-72d5194b-123e-4eeb-98f6-f2f36dd8c4af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:22.696870 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-72d5194b-123e-4eeb-98f6-f2f36dd8c4af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:22.703291 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 753/1497] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:22 2022] DELETE /identity/v3/projects/441c14c2698c4439b9d90f7e2ecfc1ad => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:22.711097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad20aa-93f3-4f9a-af92-c9cc322b855b None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:22.711539 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad20aa-93f3-4f9a-af92-c9cc322b855b None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:22.711660 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad20aa-93f3-4f9a-af92-c9cc322b855b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:22.756452 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146baa73-b24f-4a63-b5a4-3f3535c125d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:22.766051 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146baa73-b24f-4a63-b5a4-3f3535c125d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:22.766336 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146baa73-b24f-4a63-b5a4-3f3535c125d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:22.766623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146baa73-b24f-4a63-b5a4-3f3535c125d6 admin admin] PATH_INFO: `/v3/projects/9982af673646460b99e353d2392ec440/users/b2e0ea51339048e6bcc11c02f6257b03/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:22.784469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-e0ad20aa-93f3-4f9a-af92-c9cc322b855b None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:22.785740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e0ad20aa-93f3-4f9a-af92-c9cc322b855b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:22.820909 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146baa73-b24f-4a63-b5a4-3f3535c125d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9982af673646460b99e353d2392ec440, user_id=b2e0ea51339048e6bcc11c02f6257b03, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:22.823715 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146baa73-b24f-4a63-b5a4-3f3535c125d6 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:22.835335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 754/1498] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:22.837129 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dae8fc-3a48-4474-8c3c-d0fd234262d0 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:22.837420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dae8fc-3a48-4474-8c3c-d0fd234262d0 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:22.837701 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dae8fc-3a48-4474-8c3c-d0fd234262d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:22.856818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 745/1499] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:22 2022] PUT /identity/v3/projects/9982af673646460b99e353d2392ec440/users/b2e0ea51339048e6bcc11c02f6257b03/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:08:22.862621 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146baa73-b24f-4a63-b5a4-3f3535c125d6 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:22.867352 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-146baa73-b24f-4a63-b5a4-3f3535c125d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:22.882970 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-09dae8fc-3a48-4474-8c3c-d0fd234262d0 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:22.887181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-09dae8fc-3a48-4474-8c3c-d0fd234262d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:23.041387 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 755/1500] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:08:23.043170 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dae8fc-3a48-4474-8c3c-d0fd234262d0 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:23.061309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dae8fc-3a48-4474-8c3c-d0fd234262d0 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:23.075958 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e44a20-b8ff-40b9-95a2-5f223bed115f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:23.076629 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e44a20-b8ff-40b9-95a2-5f223bed115f admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:23.079202 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e44a20-b8ff-40b9-95a2-5f223bed115f admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:23.079446 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e44a20-b8ff-40b9-95a2-5f223bed115f admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:23.080191 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e44a20-b8ff-40b9-95a2-5f223bed115f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:08:23.080457 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e44a20-b8ff-40b9-95a2-5f223bed115f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:23.090538 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e44a20-b8ff-40b9-95a2-5f223bed115f admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:23.096154 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:23.098395 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146baa73-b24f-4a63-b5a4-3f3535c125d6 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:23.142106 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487436b8-92af-443c-ab22-d741f064b0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:23.148354 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487436b8-92af-443c-ab22-d741f064b0f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:23.148354 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487436b8-92af-443c-ab22-d741f064b0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:23.148629 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 756/1501] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:08:23.148629 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e44a20-b8ff-40b9-95a2-5f223bed115f admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:23.149866 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487436b8-92af-443c-ab22-d741f064b0f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:23.152897 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487436b8-92af-443c-ab22-d741f064b0f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:23.159169 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-69e44a20-b8ff-40b9-95a2-5f223bed115f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:23.161273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487436b8-92af-443c-ab22-d741f064b0f2 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:23.204246 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 746/1502] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:23.217330 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487436b8-92af-443c-ab22-d741f064b0f2 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:23.268248 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487436b8-92af-443c-ab22-d741f064b0f2 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:23.291015 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d43f2f-8f76-4b3b-8ab4-769eb6d5b830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:23.291657 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d43f2f-8f76-4b3b-8ab4-769eb6d5b830 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:23.292516 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d43f2f-8f76-4b3b-8ab4-769eb6d5b830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:23.292516 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d43f2f-8f76-4b3b-8ab4-769eb6d5b830 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:23.292879 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d43f2f-8f76-4b3b-8ab4-769eb6d5b830 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:23.294807 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d43f2f-8f76-4b3b-8ab4-769eb6d5b830 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:23.377654 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 747/1503] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:23 2022] POST /identity/v3/projects => generated 355 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:23.377654 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d43f2f-8f76-4b3b-8ab4-769eb6d5b830 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:23.421792 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e44a20-b8ff-40b9-95a2-5f223bed115f admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:23.529899 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d43f2f-8f76-4b3b-8ab4-769eb6d5b830 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:23.551113 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e897d12-1d02-436b-bef4-169d9d005dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:23.551820 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e897d12-1d02-436b-bef4-169d9d005dd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:23.552114 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e897d12-1d02-436b-bef4-169d9d005dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:23.552393 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e897d12-1d02-436b-bef4-169d9d005dd2 admin admin] PATH_INFO: `/v3/projects/9982af673646460b99e353d2392ec440/users/b2e0ea51339048e6bcc11c02f6257b03/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:23.560321 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e897d12-1d02-436b-bef4-169d9d005dd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9982af673646460b99e353d2392ec440, user_id=b2e0ea51339048e6bcc11c02f6257b03, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:23.563640 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e897d12-1d02-436b-bef4-169d9d005dd2 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:23.603000 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 748/1504] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:23 2022] PUT /identity/v3/projects/9982af673646460b99e353d2392ec440/users/b2e0ea51339048e6bcc11c02f6257b03/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:23.603846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e897d12-1d02-436b-bef4-169d9d005dd2 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:23.641692 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:23.664945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc483db-c544-4c88-8818-5bcdab72baa4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:23.667069 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc483db-c544-4c88-8818-5bcdab72baa4 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:23.670571 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc483db-c544-4c88-8818-5bcdab72baa4 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:23.673118 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc483db-c544-4c88-8818-5bcdab72baa4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:23.677946 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-8bc483db-c544-4c88-8818-5bcdab72baa4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:23.710633 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc483db-c544-4c88-8818-5bcdab72baa4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:23.712901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc483db-c544-4c88-8818-5bcdab72baa4 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:23.729463 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 757/1505] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:23.732533 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb3fbc-c755-4dd5-a862-7c29fcd93be5 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:23.733400 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb3fbc-c755-4dd5-a862-7c29fcd93be5 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:23.734662 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb3fbc-c755-4dd5-a862-7c29fcd93be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:23.793793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-c1eb3fbc-c755-4dd5-a862-7c29fcd93be5 None None] MFA Rules not processed for user `b2e0ea51339048e6bcc11c02f6257b03`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:23.795279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-c1eb3fbc-c755-4dd5-a862-7c29fcd93be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:23.887107 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e897d12-1d02-436b-bef4-169d9d005dd2 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:23.910389 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc680597-3390-4eac-876c-36a4f047b8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:23.911569 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc680597-3390-4eac-876c-36a4f047b8e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:23.911569 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc680597-3390-4eac-876c-36a4f047b8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:23.911569 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc680597-3390-4eac-876c-36a4f047b8e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:23.912346 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc680597-3390-4eac-876c-36a4f047b8e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:23.914634 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc680597-3390-4eac-876c-36a4f047b8e0 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:23.928572 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 758/1506] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:23.931047 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13dde3e-d384-4787-9e2d-9b603e6c1e1e None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:23.931353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13dde3e-d384-4787-9e2d-9b603e6c1e1e None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:23.932526 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13dde3e-d384-4787-9e2d-9b603e6c1e1e None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:23.934312 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 759/1507] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 00:08:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:08:23.940689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13dde3e-d384-4787-9e2d-9b603e6c1e1e None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:23.943268 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b13dde3e-d384-4787-9e2d-9b603e6c1e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:23.966015 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 749/1508] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:23 2022] POST /identity/v3/users => generated 278 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:23.970890 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc680597-3390-4eac-876c-36a4f047b8e0 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:23.982689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:23.991970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc680597-3390-4eac-876c-36a4f047b8e0 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:23.997208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc3de3-5ac4-424f-ac3b-eba39ab4a2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:23.998198 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc3de3-5ac4-424f-ac3b-eba39ab4a2ca admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:23.998556 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc3de3-5ac4-424f-ac3b-eba39ab4a2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:23.998834 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc3de3-5ac4-424f-ac3b-eba39ab4a2ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:24.000352 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc3de3-5ac4-424f-ac3b-eba39ab4a2ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:24.003753 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc3de3-5ac4-424f-ac3b-eba39ab4a2ca admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:24.035057 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 750/1509] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:24.038629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172a9c7-5095-4ce5-84bb-10cdd685f6cb None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:24.041464 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172a9c7-5095-4ce5-84bb-10cdd685f6cb None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:24.041902 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172a9c7-5095-4ce5-84bb-10cdd685f6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:24.101530 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-a172a9c7-5095-4ce5-84bb-10cdd685f6cb None None] MFA Rules not processed for user `c4a3579d56294b9d8ad36d4d174c8526`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:24.103345 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a172a9c7-5095-4ce5-84bb-10cdd685f6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:24.112679 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13dde3e-d384-4787-9e2d-9b603e6c1e1e None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:24.116672 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb330804-66bd-42ef-8960-dbda3a1eb087 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:24.117469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb330804-66bd-42ef-8960-dbda3a1eb087 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:24.117971 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb330804-66bd-42ef-8960-dbda3a1eb087 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:24.118352 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb330804-66bd-42ef-8960-dbda3a1eb087 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:24.143585 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb330804-66bd-42ef-8960-dbda3a1eb087 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:24.145951 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb330804-66bd-42ef-8960-dbda3a1eb087 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:24.163804 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 760/1510] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:23 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:24.168469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb330804-66bd-42ef-8960-dbda3a1eb087 service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:24.187143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb330804-66bd-42ef-8960-dbda3a1eb087 service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:24.197637 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40664d06-c8d6-4c22-ad4f-0c6401b0f248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:24.200038 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40664d06-c8d6-4c22-ad4f-0c6401b0f248 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:24.200521 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40664d06-c8d6-4c22-ad4f-0c6401b0f248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:24.200939 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40664d06-c8d6-4c22-ad4f-0c6401b0f248 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:24.207587 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40664d06-c8d6-4c22-ad4f-0c6401b0f248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:24.209932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40664d06-c8d6-4c22-ad4f-0c6401b0f248 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:24.252715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 761/1511] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:24.260175 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40664d06-c8d6-4c22-ad4f-0c6401b0f248 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:24.279515 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40664d06-c8d6-4c22-ad4f-0c6401b0f248 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:24.291474 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 751/1512] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 00:08:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:24.291802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1bfce-731d-4289-a41f-12b07c7279cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:24.293517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1bfce-731d-4289-a41f-12b07c7279cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:24.296841 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1bfce-731d-4289-a41f-12b07c7279cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:24.316661 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1bfce-731d-4289-a41f-12b07c7279cf admin admin] PATH_INFO: `/v3/projects/a572adef60f9491e89d0cc74bbde26d2/users/741fc52ad5a742cc89eb6c8253cb7f25/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:24.379076 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1bfce-731d-4289-a41f-12b07c7279cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a572adef60f9491e89d0cc74bbde26d2, user_id=741fc52ad5a742cc89eb6c8253cb7f25, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:24.393180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1bfce-731d-4289-a41f-12b07c7279cf admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:24.428113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 762/1513] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:24 2022] PUT /identity/v3/projects/a572adef60f9491e89d0cc74bbde26d2/users/741fc52ad5a742cc89eb6c8253cb7f25/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:24.443956 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552f5b9-13b4-4b77-8f42-0193488ac43e None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:24.444477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552f5b9-13b4-4b77-8f42-0193488ac43e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:24.444938 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552f5b9-13b4-4b77-8f42-0193488ac43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:24.515868 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1bfce-731d-4289-a41f-12b07c7279cf admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:24.549372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-6552f5b9-13b4-4b77-8f42-0193488ac43e None None] MFA Rules not processed for user `741fc52ad5a742cc89eb6c8253cb7f25`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:24.554788 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-6552f5b9-13b4-4b77-8f42-0193488ac43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:24.693027 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1bfce-731d-4289-a41f-12b07c7279cf admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:24.701327 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce49b0-a721-45ed-8767-1a4eca136467 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:24.703003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce49b0-a721-45ed-8767-1a4eca136467 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:24.703472 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce49b0-a721-45ed-8767-1a4eca136467 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:24.703922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce49b0-a721-45ed-8767-1a4eca136467 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:24.709489 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-84ce49b0-a721-45ed-8767-1a4eca136467 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:24.720614 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 752/1514] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:24.746101 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552f5b9-13b4-4b77-8f42-0193488ac43e None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:24.785529 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:24.903998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce49b0-a721-45ed-8767-1a4eca136467 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:24.907123 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce49b0-a721-45ed-8767-1a4eca136467 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:24.951168 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 763/1515] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:24.951168 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce49b0-a721-45ed-8767-1a4eca136467 service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:24.994400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552f5b9-13b4-4b77-8f42-0193488ac43e None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:25.028595 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884ba9f-555c-45da-b823-b74049b2587f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:25.031014 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884ba9f-555c-45da-b823-b74049b2587f admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:25.031582 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884ba9f-555c-45da-b823-b74049b2587f admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:25.032108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884ba9f-555c-45da-b823-b74049b2587f admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:25.033037 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a884ba9f-555c-45da-b823-b74049b2587f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:25.035246 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a884ba9f-555c-45da-b823-b74049b2587f admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:25.051458 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:25.169903 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 753/1516] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:24 2022] POST /identity/v3/projects => generated 357 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:25.174051 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884ba9f-555c-45da-b823-b74049b2587f admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:25.184078 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce49b0-a721-45ed-8767-1a4eca136467 service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:25.348541 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00afe2-2ab6-474d-8426-2f2393fecc90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:25.349206 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00afe2-2ab6-474d-8426-2f2393fecc90 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:25.349495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00afe2-2ab6-474d-8426-2f2393fecc90 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:25.349766 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00afe2-2ab6-474d-8426-2f2393fecc90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:25.363172 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884ba9f-555c-45da-b823-b74049b2587f admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:25.365181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00afe2-2ab6-474d-8426-2f2393fecc90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:25.367016 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00afe2-2ab6-474d-8426-2f2393fecc90 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:25.380002 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ffd8a-6b9c-4fe8-9057-6d6af1b09563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:25.381436 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ffd8a-6b9c-4fe8-9057-6d6af1b09563 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:25.382129 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ffd8a-6b9c-4fe8-9057-6d6af1b09563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:25.384003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 764/1517] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:25.384666 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ffd8a-6b9c-4fe8-9057-6d6af1b09563 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:25.385593 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ffd8a-6b9c-4fe8-9057-6d6af1b09563 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:25.392311 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ffd8a-6b9c-4fe8-9057-6d6af1b09563 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:25.396957 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00afe2-2ab6-474d-8426-2f2393fecc90 service nova] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:25.398394 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-bb00afe2-2ab6-474d-8426-2f2393fecc90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:25.424250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00afe2-2ab6-474d-8426-2f2393fecc90 service nova] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:25.434280 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef39f11-d2ad-4e10-b138-a21a5cb5aeda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:25.437696 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef39f11-d2ad-4e10-b138-a21a5cb5aeda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:25.437994 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef39f11-d2ad-4e10-b138-a21a5cb5aeda admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:25.438285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef39f11-d2ad-4e10-b138-a21a5cb5aeda admin admin] PATH_INFO: `/v3/users/9f11a38793d94bfd815581f14b07ac6b` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:25.441782 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef39f11-d2ad-4e10-b138-a21a5cb5aeda admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f11a38793d94bfd815581f14b07ac6b)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:25.445233 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef39f11-d2ad-4e10-b138-a21a5cb5aeda admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:25.455711 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 754/1518] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:25 2022] POST /identity/v3/users => generated 285 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:08:25.457201 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ffd8a-6b9c-4fe8-9057-6d6af1b09563 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:25.480969 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ffd8a-6b9c-4fe8-9057-6d6af1b09563 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:25.495401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a1e63-d39c-47ea-917f-f718b55c7286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:25.496356 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a1e63-d39c-47ea-917f-f718b55c7286 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:25.496795 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a1e63-d39c-47ea-917f-f718b55c7286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:25.497213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a1e63-d39c-47ea-917f-f718b55c7286 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:25.497999 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a1e63-d39c-47ea-917f-f718b55c7286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:25.500901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a1e63-d39c-47ea-917f-f718b55c7286 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:25.546867 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 755/1519] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:25.547379 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-cef39f11-d2ad-4e10-b138-a21a5cb5aeda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f11a38793d94bfd815581f14b07ac6b'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:25.548525 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a1e63-d39c-47ea-917f-f718b55c7286 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:25.567455 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-cef39f11-d2ad-4e10-b138-a21a5cb5aeda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f11a38793d94bfd815581f14b07ac6b'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:25.586050 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-cef39f11-d2ad-4e10-b138-a21a5cb5aeda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f11a38793d94bfd815581f14b07ac6b'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:25.587519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-cef39f11-d2ad-4e10-b138-a21a5cb5aeda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f11a38793d94bfd815581f14b07ac6b'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:25.588290 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-cef39f11-d2ad-4e10-b138-a21a5cb5aeda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f11a38793d94bfd815581f14b07ac6b'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:25.611620 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a1e63-d39c-47ea-917f-f718b55c7286 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:25.623027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72864398-2758-408d-9b93-90b73c376bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:25.623865 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72864398-2758-408d-9b93-90b73c376bd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:25.624244 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72864398-2758-408d-9b93-90b73c376bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:25.624732 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72864398-2758-408d-9b93-90b73c376bd5 admin admin] PATH_INFO: `/v3/projects/486be0966e3d42d192a16f8da75cb319/users/d22b209958844340a5a3538a2fbbfe01/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:25.658203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 765/1520] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:25 2022] DELETE /identity/v3/users/9f11a38793d94bfd815581f14b07ac6b => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:08:25.659412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef39f11-d2ad-4e10-b138-a21a5cb5aeda admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:25.661211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-cef39f11-d2ad-4e10-b138-a21a5cb5aeda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:25.668477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72864398-2758-408d-9b93-90b73c376bd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=486be0966e3d42d192a16f8da75cb319, user_id=d22b209958844340a5a3538a2fbbfe01, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:25.672799 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72864398-2758-408d-9b93-90b73c376bd5 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:25.695115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 756/1521] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:25 2022] PUT /identity/v3/projects/486be0966e3d42d192a16f8da75cb319/users/d22b209958844340a5a3538a2fbbfe01/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:25.696382 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28875029-6cdc-4408-9b94-7bad64da817e None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:25.696698 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28875029-6cdc-4408-9b94-7bad64da817e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:25.697506 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28875029-6cdc-4408-9b94-7bad64da817e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:25.727626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:25.737654 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-28875029-6cdc-4408-9b94-7bad64da817e None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:25.738899 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-28875029-6cdc-4408-9b94-7bad64da817e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:25.846633 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:25.858932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef39f11-d2ad-4e10-b138-a21a5cb5aeda admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:25.889912 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 757/1522] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:25.889912 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28875029-6cdc-4408-9b94-7bad64da817e None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:25.889912 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-28875029-6cdc-4408-9b94-7bad64da817e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:25.922204 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28875029-6cdc-4408-9b94-7bad64da817e None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:25.935999 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c3b20-50bf-43b7-aafc-16a8f0c0ed5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:25.948398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c3b20-50bf-43b7-aafc-16a8f0c0ed5f admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:25.961394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c3b20-50bf-43b7-aafc-16a8f0c0ed5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:25.964389 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c3b20-50bf-43b7-aafc-16a8f0c0ed5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:25.966028 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c3b20-50bf-43b7-aafc-16a8f0c0ed5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:25.970456 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846c3b20-50bf-43b7-aafc-16a8f0c0ed5f admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:26.007358 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 758/1523] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:26.014840 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c3b20-50bf-43b7-aafc-16a8f0c0ed5f admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:26.021397 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-846c3b20-50bf-43b7-aafc-16a8f0c0ed5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:26.023421 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d50bf9-b71f-4c25-b7c6-2f3f4a3b6e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:26.024193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d50bf9-b71f-4c25-b7c6-2f3f4a3b6e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:26.024693 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d50bf9-b71f-4c25-b7c6-2f3f4a3b6e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:26.025089 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d50bf9-b71f-4c25-b7c6-2f3f4a3b6e30 admin admin] PATH_INFO: `/v3/users/044624f790b243ee895ae65c4fa729eb` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:26.026703 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d50bf9-b71f-4c25-b7c6-2f3f4a3b6e30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=044624f790b243ee895ae65c4fa729eb)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:26.033976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d50bf9-b71f-4c25-b7c6-2f3f4a3b6e30 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:26.125557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846c3b20-50bf-43b7-aafc-16a8f0c0ed5f admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:26.136979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cefe72-6843-460c-aa6e-c42cfea743c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:26.138302 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cefe72-6843-460c-aa6e-c42cfea743c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:26.138489 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cefe72-6843-460c-aa6e-c42cfea743c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:26.138801 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cefe72-6843-460c-aa6e-c42cfea743c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:26.162229 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cefe72-6843-460c-aa6e-c42cfea743c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:26.166211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cefe72-6843-460c-aa6e-c42cfea743c8 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:26.191849 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 759/1524] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:26.192746 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cefe72-6843-460c-aa6e-c42cfea743c8 service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:26.220133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cefe72-6843-460c-aa6e-c42cfea743c8 service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:26.225349 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-78d50bf9-b71f-4c25-b7c6-2f3f4a3b6e30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '044624f790b243ee895ae65c4fa729eb'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:26.239117 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367b4b2-282b-4ce6-a394-4ea2c1707467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:26.239995 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367b4b2-282b-4ce6-a394-4ea2c1707467 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:26.240217 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367b4b2-282b-4ce6-a394-4ea2c1707467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:26.240533 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367b4b2-282b-4ce6-a394-4ea2c1707467 admin admin] PATH_INFO: `/v3/projects/486be0966e3d42d192a16f8da75cb319/users/d22b209958844340a5a3538a2fbbfe01/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:26.242493 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5367b4b2-282b-4ce6-a394-4ea2c1707467 admin admin] RBAC: Authorizing `identity:create_grant(project_id=486be0966e3d42d192a16f8da75cb319, user_id=d22b209958844340a5a3538a2fbbfe01, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:26.247845 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5367b4b2-282b-4ce6-a394-4ea2c1707467 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:26.248638 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-78d50bf9-b71f-4c25-b7c6-2f3f4a3b6e30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '044624f790b243ee895ae65c4fa729eb'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:26.275052 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 760/1525] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:26 2022] PUT /identity/v3/projects/486be0966e3d42d192a16f8da75cb319/users/d22b209958844340a5a3538a2fbbfe01/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:26.289144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48301f80-ab76-4974-8059-077a1f7af389 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:26.289507 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48301f80-ab76-4974-8059-077a1f7af389 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:26.289792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48301f80-ab76-4974-8059-077a1f7af389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:26.294005 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-78d50bf9-b71f-4c25-b7c6-2f3f4a3b6e30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '044624f790b243ee895ae65c4fa729eb'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:26.296724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-78d50bf9-b71f-4c25-b7c6-2f3f4a3b6e30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '044624f790b243ee895ae65c4fa729eb'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:26.297982 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-78d50bf9-b71f-4c25-b7c6-2f3f4a3b6e30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '044624f790b243ee895ae65c4fa729eb'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:26.338642 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 766/1526] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:25 2022] DELETE /identity/v3/users/044624f790b243ee895ae65c4fa729eb => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:26.340190 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceae033-483a-4071-8391-4cae15656674 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:26.340469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceae033-483a-4071-8391-4cae15656674 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:26.340835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceae033-483a-4071-8391-4cae15656674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:26.360667 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-48301f80-ab76-4974-8059-077a1f7af389 None None] MFA Rules not processed for user `d22b209958844340a5a3538a2fbbfe01`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:26.361927 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-48301f80-ab76-4974-8059-077a1f7af389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:26.384836 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-aceae033-483a-4071-8391-4cae15656674 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:26.387493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-aceae033-483a-4071-8391-4cae15656674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:26.539848 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 761/1527] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:26 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:26.562562 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48301f80-ab76-4974-8059-077a1f7af389 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:26.564013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-48301f80-ab76-4974-8059-077a1f7af389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:26.599705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:26.654827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 767/1528] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:26.710541 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceae033-483a-4071-8391-4cae15656674 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:26.719573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-aceae033-483a-4071-8391-4cae15656674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:26.778089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48301f80-ab76-4974-8059-077a1f7af389 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:26.793044 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cdb74e-7616-467c-941b-feabd0a903a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:26.793983 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cdb74e-7616-467c-941b-feabd0a903a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:26.794583 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cdb74e-7616-467c-941b-feabd0a903a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:26.794878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cdb74e-7616-467c-941b-feabd0a903a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:26.809615 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:26.813913 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cdb74e-7616-467c-941b-feabd0a903a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:26.818496 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cdb74e-7616-467c-941b-feabd0a903a6 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:26.847487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 762/1529] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:26 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:08:26.866362 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceae033-483a-4071-8391-4cae15656674 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:26.873605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edf63a-4a32-45f7-9b67-e8fd3673985d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:26.874637 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edf63a-4a32-45f7-9b67-e8fd3673985d service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:26.875191 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edf63a-4a32-45f7-9b67-e8fd3673985d service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:26.875694 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edf63a-4a32-45f7-9b67-e8fd3673985d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:26.894120 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01edf63a-4a32-45f7-9b67-e8fd3673985d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:26.897299 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01edf63a-4a32-45f7-9b67-e8fd3673985d service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:26.913577 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 768/1530] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:08:26.986090 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cdb74e-7616-467c-941b-feabd0a903a6 service cinder] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:26.996584 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cdb74e-7616-467c-941b-feabd0a903a6 service cinder] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:27.005368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129b50a-8ba9-4627-89e3-46995c1dbacc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:27.005984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129b50a-8ba9-4627-89e3-46995c1dbacc service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:27.006290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129b50a-8ba9-4627-89e3-46995c1dbacc service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:27.006553 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129b50a-8ba9-4627-89e3-46995c1dbacc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:27.008561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-4129b50a-8ba9-4627-89e3-46995c1dbacc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:27.087955 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edf63a-4a32-45f7-9b67-e8fd3673985d service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:27.089427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-01edf63a-4a32-45f7-9b67-e8fd3673985d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:27.135794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edf63a-4a32-45f7-9b67-e8fd3673985d service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:27.152555 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d40e8-728b-47b7-885e-adf1496c9a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:27.153220 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d40e8-728b-47b7-885e-adf1496c9a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:27.153508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d40e8-728b-47b7-885e-adf1496c9a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:27.153787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d40e8-728b-47b7-885e-adf1496c9a27 admin admin] PATH_INFO: `/v3/projects/cb9121f281484ad89e59bf20e9e0a276` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:27.156932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d40e8-728b-47b7-885e-adf1496c9a27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb9121f281484ad89e59bf20e9e0a276)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:27.160004 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d40e8-728b-47b7-885e-adf1496c9a27 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:27.191211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129b50a-8ba9-4627-89e3-46995c1dbacc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:27.193473 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129b50a-8ba9-4627-89e3-46995c1dbacc service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:27.218049 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 763/1531] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:26 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:08:27.328191 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-6b7d40e8-728b-47b7-885e-adf1496c9a27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb9121f281484ad89e59bf20e9e0a276', 'request_id': 'req-6b7d40e8-728b-47b7-885e-adf1496c9a27'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:27.347657 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-6b7d40e8-728b-47b7-885e-adf1496c9a27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb9121f281484ad89e59bf20e9e0a276', 'request_id': 'req-6b7d40e8-728b-47b7-885e-adf1496c9a27'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:27.362340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-6b7d40e8-728b-47b7-885e-adf1496c9a27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb9121f281484ad89e59bf20e9e0a276', 'request_id': 'req-6b7d40e8-728b-47b7-885e-adf1496c9a27'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:27.389131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-6b7d40e8-728b-47b7-885e-adf1496c9a27 admin admin] The token cache is being invalidate because project cb9121f281484ad89e59bf20e9e0a276 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:27.389609 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-6b7d40e8-728b-47b7-885e-adf1496c9a27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:27.390527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-6b7d40e8-728b-47b7-885e-adf1496c9a27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:27.394584 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 769/1532] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:27 2022] DELETE /identity/v3/projects/cb9121f281484ad89e59bf20e9e0a276 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:08:27.400271 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129b50a-8ba9-4627-89e3-46995c1dbacc service cinder] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:27.459495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129b50a-8ba9-4627-89e3-46995c1dbacc service cinder] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:27.557478 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b3f86-7253-4a9e-9ed2-3d8878fad903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:27.558510 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b3f86-7253-4a9e-9ed2-3d8878fad903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:27.558656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b3f86-7253-4a9e-9ed2-3d8878fad903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:27.559209 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b3f86-7253-4a9e-9ed2-3d8878fad903 admin admin] PATH_INFO: `/v3/users/69bec4d01e4243de98e18350697b7955` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:27.560421 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b3f86-7253-4a9e-9ed2-3d8878fad903 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69bec4d01e4243de98e18350697b7955)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:27.563738 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b3f86-7253-4a9e-9ed2-3d8878fad903 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:27.626109 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d40e8-728b-47b7-885e-adf1496c9a27 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:27.627813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-6b7d40e8-728b-47b7-885e-adf1496c9a27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:27.644822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-855b3f86-7253-4a9e-9ed2-3d8878fad903 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69bec4d01e4243de98e18350697b7955'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:27.645679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-855b3f86-7253-4a9e-9ed2-3d8878fad903 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69bec4d01e4243de98e18350697b7955'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:27.648349 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d40e8-728b-47b7-885e-adf1496c9a27 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:27.654554 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-855b3f86-7253-4a9e-9ed2-3d8878fad903 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69bec4d01e4243de98e18350697b7955'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:27.656037 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c49ccc6-750a-4b5c-a7c5-7b4d90bd8121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:27.657013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49ccc6-750a-4b5c-a7c5-7b4d90bd8121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:27.657368 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49ccc6-750a-4b5c-a7c5-7b4d90bd8121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:27.657712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49ccc6-750a-4b5c-a7c5-7b4d90bd8121 admin admin] PATH_INFO: `/v3/projects/08d343eb97fc46e29e2ba18751273dbf` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:27.660023 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49ccc6-750a-4b5c-a7c5-7b4d90bd8121 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08d343eb97fc46e29e2ba18751273dbf)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:27.663255 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c49ccc6-750a-4b5c-a7c5-7b4d90bd8121 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:27.672858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-855b3f86-7253-4a9e-9ed2-3d8878fad903 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69bec4d01e4243de98e18350697b7955'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:27.682964 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-855b3f86-7253-4a9e-9ed2-3d8878fad903 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69bec4d01e4243de98e18350697b7955'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:27.690384 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 764/1533] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:27 2022] DELETE /identity/v3/users/69bec4d01e4243de98e18350697b7955 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:27.730520 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1c49ccc6-750a-4b5c-a7c5-7b4d90bd8121 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08d343eb97fc46e29e2ba18751273dbf', 'request_id': 'req-1c49ccc6-750a-4b5c-a7c5-7b4d90bd8121'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:27.741959 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1c49ccc6-750a-4b5c-a7c5-7b4d90bd8121 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08d343eb97fc46e29e2ba18751273dbf', 'request_id': 'req-1c49ccc6-750a-4b5c-a7c5-7b4d90bd8121'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:27.751514 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1c49ccc6-750a-4b5c-a7c5-7b4d90bd8121 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08d343eb97fc46e29e2ba18751273dbf', 'request_id': 'req-1c49ccc6-750a-4b5c-a7c5-7b4d90bd8121'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:27.758804 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:27.767801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1c49ccc6-750a-4b5c-a7c5-7b4d90bd8121 admin admin] The token cache is being invalidate because project 08d343eb97fc46e29e2ba18751273dbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:27.768416 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1c49ccc6-750a-4b5c-a7c5-7b4d90bd8121 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:27.769898 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1c49ccc6-750a-4b5c-a7c5-7b4d90bd8121 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:27.775712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 770/1534] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:27 2022] DELETE /identity/v3/projects/08d343eb97fc46e29e2ba18751273dbf => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:27.778210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ebd3a-f776-41c9-92e4-aeb3d2e60982 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:27.778562 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ebd3a-f776-41c9-92e4-aeb3d2e60982 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:27.779120 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ebd3a-f776-41c9-92e4-aeb3d2e60982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:27.805644 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-cd6ebd3a-f776-41c9-92e4-aeb3d2e60982 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:27.806877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-cd6ebd3a-f776-41c9-92e4-aeb3d2e60982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:27.818956 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:27.892759 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 765/1535] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:27.901089 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164fa18-5cf4-4c59-a63b-219303aad63e None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:27.901583 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164fa18-5cf4-4c59-a63b-219303aad63e None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:27.901685 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164fa18-5cf4-4c59-a63b-219303aad63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:27.923565 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-f164fa18-5cf4-4c59-a63b-219303aad63e None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:27.924907 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f164fa18-5cf4-4c59-a63b-219303aad63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:27.933152 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 771/1536] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:27.937482 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ebd3a-f776-41c9-92e4-aeb3d2e60982 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:27.947712 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ebd3a-f776-41c9-92e4-aeb3d2e60982 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:27.956282 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf02d47-06e8-4950-9e7f-cd42ddfc9052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:27.957870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf02d47-06e8-4950-9e7f-cd42ddfc9052 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:27.958364 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf02d47-06e8-4950-9e7f-cd42ddfc9052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:27.958639 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf02d47-06e8-4950-9e7f-cd42ddfc9052 admin admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:27.959613 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf02d47-06e8-4950-9e7f-cd42ddfc9052 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:08:27.960055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf02d47-06e8-4950-9e7f-cd42ddfc9052 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:27.962070 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf02d47-06e8-4950-9e7f-cd42ddfc9052 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:27.975617 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 766/1537] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:08:27.979841 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164fa18-5cf4-4c59-a63b-219303aad63e None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:27.990387 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164fa18-5cf4-4c59-a63b-219303aad63e None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:27.995423 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb166f30-3531-4b54-b01c-f2a742921b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:27.996223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb166f30-3531-4b54-b01c-f2a742921b86 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:27.996607 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb166f30-3531-4b54-b01c-f2a742921b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:27.996995 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb166f30-3531-4b54-b01c-f2a742921b86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:27.997715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb166f30-3531-4b54-b01c-f2a742921b86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:28.000448 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb166f30-3531-4b54-b01c-f2a742921b86 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:28.021049 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf02d47-06e8-4950-9e7f-cd42ddfc9052 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:28.024243 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-caf02d47-06e8-4950-9e7f-cd42ddfc9052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:28.042603 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf02d47-06e8-4950-9e7f-cd42ddfc9052 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:28.049129 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c9766-8862-4cf3-8017-de8099b4f9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:28.049778 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c9766-8862-4cf3-8017-de8099b4f9bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:28.050050 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c9766-8862-4cf3-8017-de8099b4f9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:28.050331 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c9766-8862-4cf3-8017-de8099b4f9bd admin admin] PATH_INFO: `/v3/projects/679f88d0ca8b4a719a079ed0d8daf83a` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:28.050998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 772/1538] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:27 2022] POST /identity/v3/projects => generated 377 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:28.054700 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c9766-8862-4cf3-8017-de8099b4f9bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=679f88d0ca8b4a719a079ed0d8daf83a)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:28.057670 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb166f30-3531-4b54-b01c-f2a742921b86 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:28.068517 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c9766-8862-4cf3-8017-de8099b4f9bd admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:28.143420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb166f30-3531-4b54-b01c-f2a742921b86 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:28.186255 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-ee3c9766-8862-4cf3-8017-de8099b4f9bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '679f88d0ca8b4a719a079ed0d8daf83a', 'request_id': 'req-ee3c9766-8862-4cf3-8017-de8099b4f9bd'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:28.199683 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-ee3c9766-8862-4cf3-8017-de8099b4f9bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '679f88d0ca8b4a719a079ed0d8daf83a', 'request_id': 'req-ee3c9766-8862-4cf3-8017-de8099b4f9bd'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:28.207783 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-ee3c9766-8862-4cf3-8017-de8099b4f9bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '679f88d0ca8b4a719a079ed0d8daf83a', 'request_id': 'req-ee3c9766-8862-4cf3-8017-de8099b4f9bd'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:28.221500 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-ee3c9766-8862-4cf3-8017-de8099b4f9bd admin admin] The token cache is being invalidate because project 679f88d0ca8b4a719a079ed0d8daf83a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:28.221799 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-ee3c9766-8862-4cf3-8017-de8099b4f9bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:28.222496 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-ee3c9766-8862-4cf3-8017-de8099b4f9bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:28.224052 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 767/1539] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:28 2022] DELETE /identity/v3/projects/679f88d0ca8b4a719a079ed0d8daf83a => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:28.225019 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c9766-8862-4cf3-8017-de8099b4f9bd admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:28.226346 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ee3c9766-8862-4cf3-8017-de8099b4f9bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:28.229854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9be02-e3a7-49bd-84cb-5e38c561033b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:28.230731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9be02-e3a7-49bd-84cb-5e38c561033b admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:28.231274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9be02-e3a7-49bd-84cb-5e38c561033b admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:28.231784 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9be02-e3a7-49bd-84cb-5e38c561033b admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:28.232527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9be02-e3a7-49bd-84cb-5e38c561033b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:28.235725 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9be02-e3a7-49bd-84cb-5e38c561033b admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:28.241747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c9766-8862-4cf3-8017-de8099b4f9bd admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:28.248037 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3c228-b398-4e52-a761-e8560a7fbf36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:28.249433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3c228-b398-4e52-a761-e8560a7fbf36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:28.249727 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3c228-b398-4e52-a761-e8560a7fbf36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:28.250009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3c228-b398-4e52-a761-e8560a7fbf36 admin admin] PATH_INFO: `/v3/users/82555cf67aa84244b20511661549700f` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:28.251420 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3c228-b398-4e52-a761-e8560a7fbf36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82555cf67aa84244b20511661549700f)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:28.253629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3c228-b398-4e52-a761-e8560a7fbf36 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:28.264430 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 773/1540] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:28 2022] POST /identity/v3/users => generated 289 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:28.265934 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9be02-e3a7-49bd-84cb-5e38c561033b admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:28.268159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-31d9be02-e3a7-49bd-84cb-5e38c561033b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:28.284092 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9be02-e3a7-49bd-84cb-5e38c561033b admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:28.289117 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd3ee82-5b05-4f43-823a-b6883e7e140b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:28.289888 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3ee82-5b05-4f43-823a-b6883e7e140b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:28.290151 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3ee82-5b05-4f43-823a-b6883e7e140b admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:28.290428 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd3ee82-5b05-4f43-823a-b6883e7e140b admin admin] PATH_INFO: `/v3/users/921ca365cef74dec8cc6fbb3aa91e036` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:28.291663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3ee82-5b05-4f43-823a-b6883e7e140b admin admin] RBAC: Authorizing `identity:delete_user(user_id=921ca365cef74dec8cc6fbb3aa91e036)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:28.293663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd3ee82-5b05-4f43-823a-b6883e7e140b admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:28.306701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-71d3c228-b398-4e52-a761-e8560a7fbf36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82555cf67aa84244b20511661549700f'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:28.307438 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-71d3c228-b398-4e52-a761-e8560a7fbf36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82555cf67aa84244b20511661549700f'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:28.316866 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-71d3c228-b398-4e52-a761-e8560a7fbf36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82555cf67aa84244b20511661549700f'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:28.335936 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-71d3c228-b398-4e52-a761-e8560a7fbf36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82555cf67aa84244b20511661549700f'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:28.341124 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:28.346339 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-71d3c228-b398-4e52-a761-e8560a7fbf36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82555cf67aa84244b20511661549700f'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:28.354493 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 768/1541] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:28 2022] DELETE /identity/v3/users/82555cf67aa84244b20511661549700f => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:28.355332 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3c228-b398-4e52-a761-e8560a7fbf36 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:28.357672 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-71d3c228-b398-4e52-a761-e8560a7fbf36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:28.366717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1bd3ee82-5b05-4f43-823a-b6883e7e140b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '921ca365cef74dec8cc6fbb3aa91e036'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:28.378179 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1bd3ee82-5b05-4f43-823a-b6883e7e140b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '921ca365cef74dec8cc6fbb3aa91e036'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:28.380965 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:28.402014 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1bd3ee82-5b05-4f43-823a-b6883e7e140b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '921ca365cef74dec8cc6fbb3aa91e036'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:28.402967 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1bd3ee82-5b05-4f43-823a-b6883e7e140b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '921ca365cef74dec8cc6fbb3aa91e036'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:28.403684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1bd3ee82-5b05-4f43-823a-b6883e7e140b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '921ca365cef74dec8cc6fbb3aa91e036'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:28.423404 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 774/1542] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:28 2022] DELETE /identity/v3/users/921ca365cef74dec8cc6fbb3aa91e036 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:28.423404 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14340272-b3c0-47ba-8b1f-4f77967bbdff None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:28.423404 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14340272-b3c0-47ba-8b1f-4f77967bbdff None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:28.423404 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14340272-b3c0-47ba-8b1f-4f77967bbdff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:28.449432 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3c228-b398-4e52-a761-e8560a7fbf36 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:28.451883 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-14340272-b3c0-47ba-8b1f-4f77967bbdff None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:28.454102 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-14340272-b3c0-47ba-8b1f-4f77967bbdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:28.532836 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5ba2b-868f-4834-962d-598d1fd1a26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:28.533814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5ba2b-868f-4834-962d-598d1fd1a26e admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:28.534455 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5ba2b-868f-4834-962d-598d1fd1a26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:28.535021 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5ba2b-868f-4834-962d-598d1fd1a26e admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:28.535830 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d5ba2b-868f-4834-962d-598d1fd1a26e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:28.538143 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d5ba2b-868f-4834-962d-598d1fd1a26e admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:28.563893 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 769/1543] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:28.568159 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5ba2b-868f-4834-962d-598d1fd1a26e admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:28.570015 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 775/1544] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:28.571375 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f7d5ba2b-868f-4834-962d-598d1fd1a26e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:28.582959 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14340272-b3c0-47ba-8b1f-4f77967bbdff None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:28.586609 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-14340272-b3c0-47ba-8b1f-4f77967bbdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:28.589904 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5ba2b-868f-4834-962d-598d1fd1a26e admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:28.597570 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc4f95-7848-4681-831c-f2a3dade6eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:28.598200 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc4f95-7848-4681-831c-f2a3dade6eec admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:28.598930 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc4f95-7848-4681-831c-f2a3dade6eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:28.599210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc4f95-7848-4681-831c-f2a3dade6eec admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:28.599910 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc4f95-7848-4681-831c-f2a3dade6eec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:28.601552 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc4f95-7848-4681-831c-f2a3dade6eec admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:28.616980 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 770/1545] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:28.622410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc4f95-7848-4681-831c-f2a3dade6eec admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:28.654388 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc4f95-7848-4681-831c-f2a3dade6eec admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:28.663368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4788d4e-6392-4486-b9c7-68b234e07716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:28.664298 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4788d4e-6392-4486-b9c7-68b234e07716 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:28.664756 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4788d4e-6392-4486-b9c7-68b234e07716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:28.665146 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4788d4e-6392-4486-b9c7-68b234e07716 admin admin] PATH_INFO: `/v3/projects/d36d19cfc6c8483ebd1d80daf514d361/users/234acc4aa5e943118f737708fe9352f1/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:28.695533 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4788d4e-6392-4486-b9c7-68b234e07716 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d36d19cfc6c8483ebd1d80daf514d361, user_id=234acc4aa5e943118f737708fe9352f1, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:28.696040 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14340272-b3c0-47ba-8b1f-4f77967bbdff None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:28.698362 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4788d4e-6392-4486-b9c7-68b234e07716 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:28.705737 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa77bf-68ef-4aae-87a6-d318bf80e2ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:28.720342 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa77bf-68ef-4aae-87a6-d318bf80e2ec service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:28.720887 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa77bf-68ef-4aae-87a6-d318bf80e2ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:28.721336 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa77bf-68ef-4aae-87a6-d318bf80e2ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:28.742816 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 771/1546] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:28 2022] PUT /identity/v3/projects/d36d19cfc6c8483ebd1d80daf514d361/users/234acc4aa5e943118f737708fe9352f1/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:08:28.745755 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b159055-3793-41e3-b62e-7ce75862f3d2 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:28.746842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b159055-3793-41e3-b62e-7ce75862f3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:28.747158 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b159055-3793-41e3-b62e-7ce75862f3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:28.757553 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa77bf-68ef-4aae-87a6-d318bf80e2ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:28.766413 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa77bf-68ef-4aae-87a6-d318bf80e2ec service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:28.794897 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-3b159055-3793-41e3-b62e-7ce75862f3d2 None None] MFA Rules not processed for user `234acc4aa5e943118f737708fe9352f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:28.797005 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-3b159055-3793-41e3-b62e-7ce75862f3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:28.801340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 776/1547] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:28.911952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:28.934009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 772/1548] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:28.939839 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa77bf-68ef-4aae-87a6-d318bf80e2ec service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:28.953037 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa77bf-68ef-4aae-87a6-d318bf80e2ec service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:29.074191 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7271a6-785f-413c-a8cc-54ac1cd8f47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:29.074938 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7271a6-785f-413c-a8cc-54ac1cd8f47c admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:29.075374 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7271a6-785f-413c-a8cc-54ac1cd8f47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:29.075795 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7271a6-785f-413c-a8cc-54ac1cd8f47c admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:29.076882 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7271a6-785f-413c-a8cc-54ac1cd8f47c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:29.080244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7271a6-785f-413c-a8cc-54ac1cd8f47c admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:29.127913 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 777/1549] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:28 2022] POST /identity/v3/projects => generated 377 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:29.132004 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b159055-3793-41e3-b62e-7ce75862f3d2 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:29.147927 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b159055-3793-41e3-b62e-7ce75862f3d2 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:29.227977 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b7be5-a1fc-48f7-b34d-e8a1191473dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:29.228793 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b7be5-a1fc-48f7-b34d-e8a1191473dc admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:29.229474 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b7be5-a1fc-48f7-b34d-e8a1191473dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:29.229474 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b7be5-a1fc-48f7-b34d-e8a1191473dc admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:29.230136 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b7be5-a1fc-48f7-b34d-e8a1191473dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:29.232729 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b7be5-a1fc-48f7-b34d-e8a1191473dc admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:29.268531 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 773/1550] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:29 2022] POST /identity/v3/users => generated 295 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:29.272189 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7271a6-785f-413c-a8cc-54ac1cd8f47c admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:29.282419 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:29.287300 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7271a6-785f-413c-a8cc-54ac1cd8f47c admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:29.301623 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d719c-1131-4703-831d-cc3c031a28de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:29.305506 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d719c-1131-4703-831d-cc3c031a28de admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:29.305506 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d719c-1131-4703-831d-cc3c031a28de admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:29.305506 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d719c-1131-4703-831d-cc3c031a28de admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:29.305506 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4d719c-1131-4703-831d-cc3c031a28de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:29.306438 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4d719c-1131-4703-831d-cc3c031a28de admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:29.328361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 778/1551] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:29.333694 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b7be5-a1fc-48f7-b34d-e8a1191473dc admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:29.335965 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d719c-1131-4703-831d-cc3c031a28de admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:29.338088 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-6b4d719c-1131-4703-831d-cc3c031a28de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:29.346019 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b7be5-a1fc-48f7-b34d-e8a1191473dc admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:29.351370 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6a328-cb67-4a41-85b9-8d38d3daa3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:29.352097 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6a328-cb67-4a41-85b9-8d38d3daa3c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:29.352275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6a328-cb67-4a41-85b9-8d38d3daa3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:29.352544 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6a328-cb67-4a41-85b9-8d38d3daa3c8 admin admin] PATH_INFO: `/v3/projects/e197a27e5f4d45b9986adad2b26b7e50/users/d784702786964cfe84c28b590fc701d7/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:29.359905 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d719c-1131-4703-831d-cc3c031a28de admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:29.376267 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400d6c7-3381-427e-ac65-15ad2696115d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:29.377525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400d6c7-3381-427e-ac65-15ad2696115d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:29.378559 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400d6c7-3381-427e-ac65-15ad2696115d admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:29.379157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400d6c7-3381-427e-ac65-15ad2696115d admin admin] PATH_INFO: `/v3/projects/192e5a0e83e04615a4fedc0c22e4e00b` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:29.382633 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400d6c7-3381-427e-ac65-15ad2696115d admin admin] RBAC: Authorizing `identity:delete_project(project_id=192e5a0e83e04615a4fedc0c22e4e00b)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:29.383123 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6a328-cb67-4a41-85b9-8d38d3daa3c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e197a27e5f4d45b9986adad2b26b7e50, user_id=d784702786964cfe84c28b590fc701d7, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:29.383239 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400d6c7-3381-427e-ac65-15ad2696115d admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:29.385204 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e6a328-cb67-4a41-85b9-8d38d3daa3c8 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:29.404540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 774/1552] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:29 2022] PUT /identity/v3/projects/e197a27e5f4d45b9986adad2b26b7e50/users/d784702786964cfe84c28b590fc701d7/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:08:29.405325 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6a328-cb67-4a41-85b9-8d38d3daa3c8 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:29.409891 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-86e6a328-cb67-4a41-85b9-8d38d3daa3c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:29.461969 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-5400d6c7-3381-427e-ac65-15ad2696115d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '192e5a0e83e04615a4fedc0c22e4e00b', 'request_id': 'req-5400d6c7-3381-427e-ac65-15ad2696115d'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:29.471864 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-5400d6c7-3381-427e-ac65-15ad2696115d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '192e5a0e83e04615a4fedc0c22e4e00b', 'request_id': 'req-5400d6c7-3381-427e-ac65-15ad2696115d'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:29.482709 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-5400d6c7-3381-427e-ac65-15ad2696115d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '192e5a0e83e04615a4fedc0c22e4e00b', 'request_id': 'req-5400d6c7-3381-427e-ac65-15ad2696115d'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:29.495314 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-5400d6c7-3381-427e-ac65-15ad2696115d admin admin] The token cache is being invalidate because project 192e5a0e83e04615a4fedc0c22e4e00b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:29.495745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-5400d6c7-3381-427e-ac65-15ad2696115d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:29.496440 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-5400d6c7-3381-427e-ac65-15ad2696115d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:29.498082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 779/1553] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:29 2022] DELETE /identity/v3/projects/192e5a0e83e04615a4fedc0c22e4e00b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:29.499582 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400d6c7-3381-427e-ac65-15ad2696115d admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:29.503990 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5400d6c7-3381-427e-ac65-15ad2696115d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:29.516452 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6a328-cb67-4a41-85b9-8d38d3daa3c8 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:29.584166 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780e920-8eb7-4d8c-95b9-427dc9966b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:29.584846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780e920-8eb7-4d8c-95b9-427dc9966b36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:29.585129 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780e920-8eb7-4d8c-95b9-427dc9966b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:29.585405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780e920-8eb7-4d8c-95b9-427dc9966b36 admin admin] PATH_INFO: `/v3/projects/a0a5ea0c39c24ce4acdf4a4626999a57` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:29.586058 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400d6c7-3381-427e-ac65-15ad2696115d admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:29.587106 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780e920-8eb7-4d8c-95b9-427dc9966b36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0a5ea0c39c24ce4acdf4a4626999a57)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:29.589256 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780e920-8eb7-4d8c-95b9-427dc9966b36 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:29.592591 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17819e7-a650-48d5-a530-c45cb60b9b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:29.594106 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17819e7-a650-48d5-a530-c45cb60b9b38 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:29.594536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17819e7-a650-48d5-a530-c45cb60b9b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:29.594817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17819e7-a650-48d5-a530-c45cb60b9b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:29.595712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17819e7-a650-48d5-a530-c45cb60b9b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:29.597338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17819e7-a650-48d5-a530-c45cb60b9b38 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:29.600795 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:29.617782 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 780/1554] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:29.617782 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc33315-088d-46d8-8bb2-f9a4f9822eb2 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:29.617782 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc33315-088d-46d8-8bb2-f9a4f9822eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:29.617782 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc33315-088d-46d8-8bb2-f9a4f9822eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:29.643026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-6bc33315-088d-46d8-8bb2-f9a4f9822eb2 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:29.644363 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-6bc33315-088d-46d8-8bb2-f9a4f9822eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:29.659064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b780e920-8eb7-4d8c-95b9-427dc9966b36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0a5ea0c39c24ce4acdf4a4626999a57', 'request_id': 'req-b780e920-8eb7-4d8c-95b9-427dc9966b36'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:29.666760 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b780e920-8eb7-4d8c-95b9-427dc9966b36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0a5ea0c39c24ce4acdf4a4626999a57', 'request_id': 'req-b780e920-8eb7-4d8c-95b9-427dc9966b36'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:29.673359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b780e920-8eb7-4d8c-95b9-427dc9966b36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0a5ea0c39c24ce4acdf4a4626999a57', 'request_id': 'req-b780e920-8eb7-4d8c-95b9-427dc9966b36'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:29.684735 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b780e920-8eb7-4d8c-95b9-427dc9966b36 admin admin] The token cache is being invalidate because project a0a5ea0c39c24ce4acdf4a4626999a57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:29.685027 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b780e920-8eb7-4d8c-95b9-427dc9966b36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:29.685695 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b780e920-8eb7-4d8c-95b9-427dc9966b36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:29.687553 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 775/1555] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:29 2022] DELETE /identity/v3/projects/a0a5ea0c39c24ce4acdf4a4626999a57 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:29.688302 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780e920-8eb7-4d8c-95b9-427dc9966b36 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:29.689732 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b780e920-8eb7-4d8c-95b9-427dc9966b36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:29.707453 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 781/1556] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:29.708334 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc33315-088d-46d8-8bb2-f9a4f9822eb2 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:29.709776 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-6bc33315-088d-46d8-8bb2-f9a4f9822eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:29.722702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc33315-088d-46d8-8bb2-f9a4f9822eb2 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:29.727590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7c1b6-ea3d-4690-90e7-46505d97517a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:29.728218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7c1b6-ea3d-4690-90e7-46505d97517a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:29.728511 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7c1b6-ea3d-4690-90e7-46505d97517a admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:29.728813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7c1b6-ea3d-4690-90e7-46505d97517a admin admin] PATH_INFO: `/v3/users/9accccf5dcf3431ca82cb13493f9d58c` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:29.729946 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7c1b6-ea3d-4690-90e7-46505d97517a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9accccf5dcf3431ca82cb13493f9d58c)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:29.731846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7c1b6-ea3d-4690-90e7-46505d97517a admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:29.737736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:29.760636 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780e920-8eb7-4d8c-95b9-427dc9966b36 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:29.766308 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aec2ef-e4de-41c8-b1c4-3e015fbf05bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:29.766961 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aec2ef-e4de-41c8-b1c4-3e015fbf05bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:29.767241 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aec2ef-e4de-41c8-b1c4-3e015fbf05bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:29.767516 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aec2ef-e4de-41c8-b1c4-3e015fbf05bf admin admin] PATH_INFO: `/v3/projects/e197a27e5f4d45b9986adad2b26b7e50/users/d784702786964cfe84c28b590fc701d7/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:29.769683 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aec2ef-e4de-41c8-b1c4-3e015fbf05bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e197a27e5f4d45b9986adad2b26b7e50, user_id=d784702786964cfe84c28b590fc701d7, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:29.772518 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aec2ef-e4de-41c8-b1c4-3e015fbf05bf admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:29.783333 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 776/1557] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:29 2022] PUT /identity/v3/projects/e197a27e5f4d45b9986adad2b26b7e50/users/d784702786964cfe84c28b590fc701d7/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:29.784729 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a9a8f-f8e4-4eee-b51a-8e8a1797f92f None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:29.785112 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a9a8f-f8e4-4eee-b51a-8e8a1797f92f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:29.785504 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a9a8f-f8e4-4eee-b51a-8e8a1797f92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:29.795734 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-cbb7c1b6-ea3d-4690-90e7-46505d97517a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9accccf5dcf3431ca82cb13493f9d58c'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:29.803478 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-cbb7c1b6-ea3d-4690-90e7-46505d97517a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9accccf5dcf3431ca82cb13493f9d58c'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:29.808178 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-852a9a8f-f8e4-4eee-b51a-8e8a1797f92f None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:29.809432 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-852a9a8f-f8e4-4eee-b51a-8e8a1797f92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:29.816703 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-cbb7c1b6-ea3d-4690-90e7-46505d97517a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9accccf5dcf3431ca82cb13493f9d58c'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:29.817631 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-cbb7c1b6-ea3d-4690-90e7-46505d97517a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9accccf5dcf3431ca82cb13493f9d58c'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:29.818343 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-cbb7c1b6-ea3d-4690-90e7-46505d97517a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9accccf5dcf3431ca82cb13493f9d58c'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:29.836630 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 782/1558] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:29 2022] DELETE /identity/v3/users/9accccf5dcf3431ca82cb13493f9d58c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:29.838198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaddadf-9894-4f9a-9159-135db43cf247 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:29.838584 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaddadf-9894-4f9a-9159-135db43cf247 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:29.838958 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaddadf-9894-4f9a-9159-135db43cf247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:29.877579 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-deaddadf-9894-4f9a-9159-135db43cf247 None None] MFA Rules not processed for user `d784702786964cfe84c28b590fc701d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:29.878810 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-deaddadf-9894-4f9a-9159-135db43cf247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:29.897319 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 777/1559] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:29.901993 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a9a8f-f8e4-4eee-b51a-8e8a1797f92f None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:29.914699 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a9a8f-f8e4-4eee-b51a-8e8a1797f92f None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:29.954461 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:29.969063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 783/1560] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:29.981231 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaddadf-9894-4f9a-9159-135db43cf247 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:29.982869 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-deaddadf-9894-4f9a-9159-135db43cf247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:30.021801 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ebb62-8a7d-4812-8f73-c89b32847033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:30.022436 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ebb62-8a7d-4812-8f73-c89b32847033 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:30.022720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ebb62-8a7d-4812-8f73-c89b32847033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:30.022998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ebb62-8a7d-4812-8f73-c89b32847033 admin admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:30.023658 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ebb62-8a7d-4812-8f73-c89b32847033 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:08:30.024076 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ebb62-8a7d-4812-8f73-c89b32847033 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:30.025754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ebb62-8a7d-4812-8f73-c89b32847033 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:30.045255 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 778/1561] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:08:30.049355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ebb62-8a7d-4812-8f73-c89b32847033 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:30.067261 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ebb62-8a7d-4812-8f73-c89b32847033 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:30.075836 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9180ba-4864-4703-9eff-689549f864ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:30.078897 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9180ba-4864-4703-9eff-689549f864ad admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:30.079246 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9180ba-4864-4703-9eff-689549f864ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:30.079533 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9180ba-4864-4703-9eff-689549f864ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:30.080235 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaddadf-9894-4f9a-9159-135db43cf247 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:30.080734 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9180ba-4864-4703-9eff-689549f864ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:30.082843 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9180ba-4864-4703-9eff-689549f864ad admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:30.085347 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98432830-bd8e-4379-94f2-16c2a62fb2f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:30.085971 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98432830-bd8e-4379-94f2-16c2a62fb2f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:30.086252 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98432830-bd8e-4379-94f2-16c2a62fb2f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:30.086832 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98432830-bd8e-4379-94f2-16c2a62fb2f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:30.088688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-98432830-bd8e-4379-94f2-16c2a62fb2f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:30.121039 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 779/1562] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:30 2022] POST /identity/v3/projects => generated 377 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:08:30.125974 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9180ba-4864-4703-9eff-689549f864ad admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:30.128568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:30.189126 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98432830-bd8e-4379-94f2-16c2a62fb2f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:30.190779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98432830-bd8e-4379-94f2-16c2a62fb2f6 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:30.203093 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 784/1563] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:08:30.213920 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98432830-bd8e-4379-94f2-16c2a62fb2f6 service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:30.216530 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-98432830-bd8e-4379-94f2-16c2a62fb2f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:30.232648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9180ba-4864-4703-9eff-689549f864ad admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:30.239048 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98432830-bd8e-4379-94f2-16c2a62fb2f6 service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:30.240157 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27275bc5-c3c2-457a-8454-452f1761a2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:30.240932 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27275bc5-c3c2-457a-8454-452f1761a2af admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:30.241287 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27275bc5-c3c2-457a-8454-452f1761a2af admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:30.241813 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27275bc5-c3c2-457a-8454-452f1761a2af admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:30.242528 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27275bc5-c3c2-457a-8454-452f1761a2af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:30.245387 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27275bc5-c3c2-457a-8454-452f1761a2af admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:30.261657 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4e809-bd93-42eb-b1b9-fcccd9d52463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:30.265565 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4e809-bd93-42eb-b1b9-fcccd9d52463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:30.265858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4e809-bd93-42eb-b1b9-fcccd9d52463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:30.266159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4e809-bd93-42eb-b1b9-fcccd9d52463 admin admin] PATH_INFO: `/v3/projects/5031ec41c22d40e680521e0d3621292e` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:30.269197 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4e809-bd93-42eb-b1b9-fcccd9d52463 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5031ec41c22d40e680521e0d3621292e)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:30.275006 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 780/1564] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:30 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:30.276608 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4e809-bd93-42eb-b1b9-fcccd9d52463 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:30.279583 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27275bc5-c3c2-457a-8454-452f1761a2af admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:30.308706 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27275bc5-c3c2-457a-8454-452f1761a2af admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:30.336195 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d72f17-dfbf-4e47-8633-fc2578a4cb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:30.337031 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d72f17-dfbf-4e47-8633-fc2578a4cb51 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:30.337319 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d72f17-dfbf-4e47-8633-fc2578a4cb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:30.337733 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d72f17-dfbf-4e47-8633-fc2578a4cb51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:30.338712 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d72f17-dfbf-4e47-8633-fc2578a4cb51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:30.340333 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d72f17-dfbf-4e47-8633-fc2578a4cb51 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:30.354516 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 781/1565] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:30.362365 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d72f17-dfbf-4e47-8633-fc2578a4cb51 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:30.376463 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d72f17-dfbf-4e47-8633-fc2578a4cb51 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:30.385304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-38d4e809-bd93-42eb-b1b9-fcccd9d52463 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5031ec41c22d40e680521e0d3621292e', 'request_id': 'req-38d4e809-bd93-42eb-b1b9-fcccd9d52463'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:30.396344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-38d4e809-bd93-42eb-b1b9-fcccd9d52463 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5031ec41c22d40e680521e0d3621292e', 'request_id': 'req-38d4e809-bd93-42eb-b1b9-fcccd9d52463'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:30.413113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-38d4e809-bd93-42eb-b1b9-fcccd9d52463 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5031ec41c22d40e680521e0d3621292e', 'request_id': 'req-38d4e809-bd93-42eb-b1b9-fcccd9d52463'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:30.442191 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-38d4e809-bd93-42eb-b1b9-fcccd9d52463 admin admin] The token cache is being invalidate because project 5031ec41c22d40e680521e0d3621292e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:30.442676 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-38d4e809-bd93-42eb-b1b9-fcccd9d52463 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:30.443596 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-38d4e809-bd93-42eb-b1b9-fcccd9d52463 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:30.449215 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 785/1566] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:30 2022] DELETE /identity/v3/projects/5031ec41c22d40e680521e0d3621292e => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:08:30.451727 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f62ad-dc74-4dc6-8aed-006a322ef4e9 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:30.453179 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f62ad-dc74-4dc6-8aed-006a322ef4e9 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:30.453483 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f62ad-dc74-4dc6-8aed-006a322ef4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:30.505800 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479d91d-fac2-46ad-bc50-e53c65190b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:30.506716 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479d91d-fac2-46ad-bc50-e53c65190b7c admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:30.507016 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479d91d-fac2-46ad-bc50-e53c65190b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:30.507489 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479d91d-fac2-46ad-bc50-e53c65190b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:30.507844 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-a58f62ad-dc74-4dc6-8aed-006a322ef4e9 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:30.508387 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9479d91d-fac2-46ad-bc50-e53c65190b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:30.509463 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a58f62ad-dc74-4dc6-8aed-006a322ef4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:30.510419 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9479d91d-fac2-46ad-bc50-e53c65190b7c admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:30.523566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 782/1567] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:30.527701 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 786/1568] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:30.535188 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479d91d-fac2-46ad-bc50-e53c65190b7c admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:30.536956 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9479d91d-fac2-46ad-bc50-e53c65190b7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:30.537467 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca5f6c-b5c6-4b68-bfe3-7d070033478b None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:30.538034 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca5f6c-b5c6-4b68-bfe3-7d070033478b None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:30.538418 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca5f6c-b5c6-4b68-bfe3-7d070033478b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:30.558843 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479d91d-fac2-46ad-bc50-e53c65190b7c admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:30.565100 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3cfdd-1314-4a1c-8a81-dc97aca25b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:30.565783 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3cfdd-1314-4a1c-8a81-dc97aca25b75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:30.566089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3cfdd-1314-4a1c-8a81-dc97aca25b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:30.566338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3cfdd-1314-4a1c-8a81-dc97aca25b75 admin admin] PATH_INFO: `/v3/projects/dee07d424cad4b16b048043e765cd42c/users/4d187ce9dfb648a9abc51e3ee3f464b6/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:30.567707 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-7dca5f6c-b5c6-4b68-bfe3-7d070033478b None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:30.569506 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-7dca5f6c-b5c6-4b68-bfe3-7d070033478b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:30.581084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 787/1569] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:30.582100 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3cfdd-1314-4a1c-8a81-dc97aca25b75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dee07d424cad4b16b048043e765cd42c, user_id=4d187ce9dfb648a9abc51e3ee3f464b6, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:30.585081 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3cfdd-1314-4a1c-8a81-dc97aca25b75 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:30.585225 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca5f6c-b5c6-4b68-bfe3-7d070033478b None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:30.588438 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:30.597008 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca5f6c-b5c6-4b68-bfe3-7d070033478b None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:30.606866 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 783/1570] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:30 2022] PUT /identity/v3/projects/dee07d424cad4b16b048043e765cd42c/users/4d187ce9dfb648a9abc51e3ee3f464b6/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:30.619529 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aba59f-e6b4-4006-bc67-5132e28ebe55 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:30.619972 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aba59f-e6b4-4006-bc67-5132e28ebe55 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:30.620124 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aba59f-e6b4-4006-bc67-5132e28ebe55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:30.656343 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-18aba59f-e6b4-4006-bc67-5132e28ebe55 None None] MFA Rules not processed for user `4d187ce9dfb648a9abc51e3ee3f464b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:30.657959 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-18aba59f-e6b4-4006-bc67-5132e28ebe55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:30.685723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bc75b-a76a-48e6-8dd5-561a0b09d9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:30.686355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bc75b-a76a-48e6-8dd5-561a0b09d9f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:30.687905 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bc75b-a76a-48e6-8dd5-561a0b09d9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:30.688186 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bc75b-a76a-48e6-8dd5-561a0b09d9f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:30.689051 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390bc75b-a76a-48e6-8dd5-561a0b09d9f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:08:30.689367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390bc75b-a76a-48e6-8dd5-561a0b09d9f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:30.692304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390bc75b-a76a-48e6-8dd5-561a0b09d9f4 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:30.736379 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 788/1571] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:08:30.751894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bc75b-a76a-48e6-8dd5-561a0b09d9f4 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:30.754445 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 784/1572] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:30.758319 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aba59f-e6b4-4006-bc67-5132e28ebe55 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:30.779386 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bc75b-a76a-48e6-8dd5-561a0b09d9f4 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:30.784893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d99c05-d839-4c60-8f14-37345dd7e458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:30.785641 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d99c05-d839-4c60-8f14-37345dd7e458 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:30.786980 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d99c05-d839-4c60-8f14-37345dd7e458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:30.787257 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d99c05-d839-4c60-8f14-37345dd7e458 admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:30.787903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d99c05-d839-4c60-8f14-37345dd7e458 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:30.788377 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aba59f-e6b4-4006-bc67-5132e28ebe55 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:30.791088 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d99c05-d839-4c60-8f14-37345dd7e458 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:30.800172 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3ee87-8114-45cd-be54-b58aeadd6157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:30.801055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3ee87-8114-45cd-be54-b58aeadd6157 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:30.801417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3ee87-8114-45cd-be54-b58aeadd6157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:30.801792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3ee87-8114-45cd-be54-b58aeadd6157 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:30.802538 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a3ee87-8114-45cd-be54-b58aeadd6157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:30.805545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a3ee87-8114-45cd-be54-b58aeadd6157 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:30.849253 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 789/1573] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:30 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:30.852591 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d99c05-d839-4c60-8f14-37345dd7e458 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:30.863865 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:30.865520 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 785/1574] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:30 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:30.868781 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3ee87-8114-45cd-be54-b58aeadd6157 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:30.944673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d99c05-d839-4c60-8f14-37345dd7e458 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:30.959389 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3ee87-8114-45cd-be54-b58aeadd6157 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:30.964316 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0a440-3960-4e42-b816-257384f196f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:30.965022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0a440-3960-4e42-b816-257384f196f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:30.965303 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0a440-3960-4e42-b816-257384f196f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:30.965637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a0a440-3960-4e42-b816-257384f196f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:30.966209 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a0a440-3960-4e42-b816-257384f196f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:30.969031 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a0a440-3960-4e42-b816-257384f196f0 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:30.993044 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 786/1575] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:30 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:30.994072 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0a440-3960-4e42-b816-257384f196f0 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:30.997971 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e66765-6dde-4b2f-b410-a949cda1352a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:30.998621 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e66765-6dde-4b2f-b410-a949cda1352a admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:30.998933 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e66765-6dde-4b2f-b410-a949cda1352a admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:31.000510 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e66765-6dde-4b2f-b410-a949cda1352a admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:31.001173 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e66765-6dde-4b2f-b410-a949cda1352a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:31.003087 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e66765-6dde-4b2f-b410-a949cda1352a admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:31.004000 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a0a440-3960-4e42-b816-257384f196f0 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:31.008601 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb5da8-70a3-42d4-9eaf-92a959b3c9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:31.009188 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb5da8-70a3-42d4-9eaf-92a959b3c9c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:31.009472 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb5da8-70a3-42d4-9eaf-92a959b3c9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:31.009748 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb5da8-70a3-42d4-9eaf-92a959b3c9c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:31.010382 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb5da8-70a3-42d4-9eaf-92a959b3c9c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:31.011993 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb5da8-70a3-42d4-9eaf-92a959b3c9c7 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:31.023426 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 787/1576] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:08:31.027801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 790/1577] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:30 2022] POST /identity/v3/users => generated 294 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:31.028866 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb5da8-70a3-42d4-9eaf-92a959b3c9c7 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:31.028984 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e66765-6dde-4b2f-b410-a949cda1352a admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:31.038832 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb5da8-70a3-42d4-9eaf-92a959b3c9c7 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:31.038978 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e66765-6dde-4b2f-b410-a949cda1352a admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:31.043452 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a068228-6175-4e85-a953-8a43aef4dc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:31.043945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d8484-2ab6-4b82-ac7d-349da92b62ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:31.044075 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a068228-6175-4e85-a953-8a43aef4dc0a admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:31.044315 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a068228-6175-4e85-a953-8a43aef4dc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:31.044555 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d8484-2ab6-4b82-ac7d-349da92b62ed admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:31.044666 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a068228-6175-4e85-a953-8a43aef4dc0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:31.044865 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d8484-2ab6-4b82-ac7d-349da92b62ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:31.045140 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d8484-2ab6-4b82-ac7d-349da92b62ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:31.045252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a068228-6175-4e85-a953-8a43aef4dc0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:31.045807 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d8484-2ab6-4b82-ac7d-349da92b62ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:31.047079 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a068228-6175-4e85-a953-8a43aef4dc0a admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:31.047945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6d8484-2ab6-4b82-ac7d-349da92b62ed admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:31.057556 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 788/1578] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:31.060628 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 791/1579] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:31.062221 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d8484-2ab6-4b82-ac7d-349da92b62ed admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:31.062748 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a068228-6175-4e85-a953-8a43aef4dc0a admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:31.071942 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d8484-2ab6-4b82-ac7d-349da92b62ed admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:31.072285 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a068228-6175-4e85-a953-8a43aef4dc0a admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:31.076590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11036c-f5d2-4856-8fd2-fa5e0f71ef3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:31.076997 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0eba45-8fb1-4fad-95a3-422518fcbd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:31.077378 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11036c-f5d2-4856-8fd2-fa5e0f71ef3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:31.077982 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11036c-f5d2-4856-8fd2-fa5e0f71ef3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:31.077982 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11036c-f5d2-4856-8fd2-fa5e0f71ef3f admin admin] PATH_INFO: `/v3/projects/237d8f3f57be4b0fb27f8392808f867b/users/bc211296568947f69bab800cbbd1aaed/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:31.078148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0eba45-8fb1-4fad-95a3-422518fcbd78 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:31.078447 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0eba45-8fb1-4fad-95a3-422518fcbd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:31.078788 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0eba45-8fb1-4fad-95a3-422518fcbd78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:31.079496 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0eba45-8fb1-4fad-95a3-422518fcbd78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:31.081112 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0eba45-8fb1-4fad-95a3-422518fcbd78 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:31.090996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 789/1580] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:31.094048 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11036c-f5d2-4856-8fd2-fa5e0f71ef3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=237d8f3f57be4b0fb27f8392808f867b, user_id=bc211296568947f69bab800cbbd1aaed, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:31.096670 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0eba45-8fb1-4fad-95a3-422518fcbd78 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:31.097075 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11036c-f5d2-4856-8fd2-fa5e0f71ef3f admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:31.107287 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0eba45-8fb1-4fad-95a3-422518fcbd78 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:31.113271 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 792/1581] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:31 2022] PUT /identity/v3/projects/237d8f3f57be4b0fb27f8392808f867b/users/bc211296568947f69bab800cbbd1aaed/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:31.121335 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:31.123645 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de72ce-9337-4ae7-af3c-ef065efad76e None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:31.123970 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de72ce-9337-4ae7-af3c-ef065efad76e None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:31.124316 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de72ce-9337-4ae7-af3c-ef065efad76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:31.154609 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-f3de72ce-9337-4ae7-af3c-ef065efad76e None None] MFA Rules not processed for user `bc211296568947f69bab800cbbd1aaed`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:31.156369 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f3de72ce-9337-4ae7-af3c-ef065efad76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:31.167938 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d616b0-561e-4157-87f1-51a3b7a8f68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:31.168624 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d616b0-561e-4157-87f1-51a3b7a8f68f admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:31.168905 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d616b0-561e-4157-87f1-51a3b7a8f68f admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:31.169186 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d616b0-561e-4157-87f1-51a3b7a8f68f admin admin] PATH_INFO: `/v3/projects/38e129eab49f4b34973e8b7da5c36bfc/users/6d1031f083614c4c8847d1284d30977f/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:31.183819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d616b0-561e-4157-87f1-51a3b7a8f68f admin admin] RBAC: Authorizing `identity:create_grant(project_id=38e129eab49f4b34973e8b7da5c36bfc, user_id=6d1031f083614c4c8847d1284d30977f, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:31.186493 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d616b0-561e-4157-87f1-51a3b7a8f68f admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:31.199500 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 790/1582] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:31 2022] PUT /identity/v3/projects/38e129eab49f4b34973e8b7da5c36bfc/users/6d1031f083614c4c8847d1284d30977f/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:08:31.210230 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0bdab-bd18-4f73-9cc7-6413271ad7bc None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:31.210516 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0bdab-bd18-4f73-9cc7-6413271ad7bc None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:31.210821 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0bdab-bd18-4f73-9cc7-6413271ad7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:31.219523 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:31.226178 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 793/1583] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:08:31.230801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de72ce-9337-4ae7-af3c-ef065efad76e None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:31.241791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-dca0bdab-bd18-4f73-9cc7-6413271ad7bc None None] MFA Rules not processed for user `6d1031f083614c4c8847d1284d30977f`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:31.242648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de72ce-9337-4ae7-af3c-ef065efad76e None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:31.243028 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-dca0bdab-bd18-4f73-9cc7-6413271ad7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:31.302434 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d245d1f-f2b7-4960-a3f1-930952361998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:31.303049 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d245d1f-f2b7-4960-a3f1-930952361998 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:31.303330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d245d1f-f2b7-4960-a3f1-930952361998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:31.303602 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d245d1f-f2b7-4960-a3f1-930952361998 admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:31.304264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d245d1f-f2b7-4960-a3f1-930952361998 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:31.306186 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d245d1f-f2b7-4960-a3f1-930952361998 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:31.308334 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 791/1584] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:08:31.313919 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0bdab-bd18-4f73-9cc7-6413271ad7bc None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:31.325806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0bdab-bd18-4f73-9cc7-6413271ad7bc None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:31.332003 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e031ba-9453-445c-8272-d6e391714dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:31.332648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e031ba-9453-445c-8272-d6e391714dea admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:31.332932 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e031ba-9453-445c-8272-d6e391714dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:31.333210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e031ba-9453-445c-8272-d6e391714dea admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:31.333873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e031ba-9453-445c-8272-d6e391714dea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:31.335768 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e031ba-9453-445c-8272-d6e391714dea admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:31.335960 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 794/1585] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:31 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:08:31.339967 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d245d1f-f2b7-4960-a3f1-930952361998 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:31.367903 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 792/1586] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:31 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:08:31.369234 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e031ba-9453-445c-8272-d6e391714dea admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:31.408472 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d245d1f-f2b7-4960-a3f1-930952361998 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:31.433616 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e031ba-9453-445c-8272-d6e391714dea admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:31.439225 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f30cc-a5cb-4356-86fc-b55d9a743c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:31.439870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f30cc-a5cb-4356-86fc-b55d9a743c13 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:31.440153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f30cc-a5cb-4356-86fc-b55d9a743c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:31.440430 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f30cc-a5cb-4356-86fc-b55d9a743c13 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:31.441097 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f30cc-a5cb-4356-86fc-b55d9a743c13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:31.443037 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f30cc-a5cb-4356-86fc-b55d9a743c13 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:31.460853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:31.472087 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 793/1587] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:31 2022] POST /identity/v3/users => generated 301 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:08:31.473192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f30cc-a5cb-4356-86fc-b55d9a743c13 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:31.475186 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72657f92-12cd-46cf-914e-574b327c5940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:31.475903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72657f92-12cd-46cf-914e-574b327c5940 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:31.476285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72657f92-12cd-46cf-914e-574b327c5940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:31.476692 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72657f92-12cd-46cf-914e-574b327c5940 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:31.477378 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72657f92-12cd-46cf-914e-574b327c5940 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:31.480202 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72657f92-12cd-46cf-914e-574b327c5940 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:31.486939 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f30cc-a5cb-4356-86fc-b55d9a743c13 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:31.493843 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e6153-5541-4db4-b9d5-d6d422a970a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:31.494575 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e6153-5541-4db4-b9d5-d6d422a970a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:31.495069 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e6153-5541-4db4-b9d5-d6d422a970a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:31.495387 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e6153-5541-4db4-b9d5-d6d422a970a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:31.496111 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e6153-5541-4db4-b9d5-d6d422a970a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:31.498338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e6153-5541-4db4-b9d5-d6d422a970a6 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:31.509293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 795/1588] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:31 2022] POST /identity/v3/users => generated 294 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:08:31.514498 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 794/1589] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:31.514696 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72657f92-12cd-46cf-914e-574b327c5940 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:31.515540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e6153-5541-4db4-b9d5-d6d422a970a6 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:31.524901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72657f92-12cd-46cf-914e-574b327c5940 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:31.526463 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e6153-5541-4db4-b9d5-d6d422a970a6 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:31.530605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d876b6-dc90-4016-848e-10ac3f56111d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:31.531250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d876b6-dc90-4016-848e-10ac3f56111d admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:31.531514 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef3639-1515-4d71-bd04-6c1aa8efb6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:31.531636 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d876b6-dc90-4016-848e-10ac3f56111d admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:31.531828 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d876b6-dc90-4016-848e-10ac3f56111d admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:31.532296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef3639-1515-4d71-bd04-6c1aa8efb6f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:31.532468 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d876b6-dc90-4016-848e-10ac3f56111d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:31.534200 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef3639-1515-4d71-bd04-6c1aa8efb6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:31.534321 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d876b6-dc90-4016-848e-10ac3f56111d admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:31.534595 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef3639-1515-4d71-bd04-6c1aa8efb6f7 admin admin] PATH_INFO: `/v3/projects/095784c37a084530905c1d115805d267/users/35171261f21b44b2aa06bb6b38b1c46c/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:31.551184 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 796/1590] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:31.551882 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef3639-1515-4d71-bd04-6c1aa8efb6f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=095784c37a084530905c1d115805d267, user_id=35171261f21b44b2aa06bb6b38b1c46c, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:31.552066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d876b6-dc90-4016-848e-10ac3f56111d admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:31.557321 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ef3639-1515-4d71-bd04-6c1aa8efb6f7 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:31.564503 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d876b6-dc90-4016-848e-10ac3f56111d admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:31.570145 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de701aa-2f5d-4cde-bdb5-a8dab64e5bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:31.570918 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de701aa-2f5d-4cde-bdb5-a8dab64e5bf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:31.571385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de701aa-2f5d-4cde-bdb5-a8dab64e5bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:31.571714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de701aa-2f5d-4cde-bdb5-a8dab64e5bf0 admin admin] PATH_INFO: `/v3/projects/ee52f9e94e4b4c289a4a31c10d0905d4/users/f363a0851c4f46fb9322a5f14108b8a5/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:31.576265 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:31.579458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 795/1591] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:31 2022] PUT /identity/v3/projects/095784c37a084530905c1d115805d267/users/35171261f21b44b2aa06bb6b38b1c46c/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:31.584535 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef3639-1515-4d71-bd04-6c1aa8efb6f7 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:31.597874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de701aa-2f5d-4cde-bdb5-a8dab64e5bf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee52f9e94e4b4c289a4a31c10d0905d4, user_id=f363a0851c4f46fb9322a5f14108b8a5, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:31.602571 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de701aa-2f5d-4cde-bdb5-a8dab64e5bf0 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:31.621203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 797/1592] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:31 2022] PUT /identity/v3/projects/ee52f9e94e4b4c289a4a31c10d0905d4/users/f363a0851c4f46fb9322a5f14108b8a5/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:31.622111 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de701aa-2f5d-4cde-bdb5-a8dab64e5bf0 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:31.672792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef3639-1515-4d71-bd04-6c1aa8efb6f7 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:31.712780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de701aa-2f5d-4cde-bdb5-a8dab64e5bf0 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:31.718531 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb771878-a63e-491c-94a5-a0e10b8f6cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:31.719142 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb771878-a63e-491c-94a5-a0e10b8f6cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:31.719428 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb771878-a63e-491c-94a5-a0e10b8f6cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:31.719702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb771878-a63e-491c-94a5-a0e10b8f6cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:31.720372 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb771878-a63e-491c-94a5-a0e10b8f6cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:31.722063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb771878-a63e-491c-94a5-a0e10b8f6cf1 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:31.736560 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 798/1593] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:31.737779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb771878-a63e-491c-94a5-a0e10b8f6cf1 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:31.740808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fc2e0-1ce1-471d-b0ed-0bc6f0d854cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:31.741487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fc2e0-1ce1-471d-b0ed-0bc6f0d854cd admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:31.741794 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fc2e0-1ce1-471d-b0ed-0bc6f0d854cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:31.742183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fc2e0-1ce1-471d-b0ed-0bc6f0d854cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:31.742832 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0fc2e0-1ce1-471d-b0ed-0bc6f0d854cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:31.744632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0fc2e0-1ce1-471d-b0ed-0bc6f0d854cd admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:31.750110 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb771878-a63e-491c-94a5-a0e10b8f6cf1 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:31.756054 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0026cf-bb69-44c8-b8e1-269da5c4b98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:31.756747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0026cf-bb69-44c8-b8e1-269da5c4b98b admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:31.757035 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0026cf-bb69-44c8-b8e1-269da5c4b98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:31.757330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0026cf-bb69-44c8-b8e1-269da5c4b98b admin admin] PATH_INFO: `/v3/projects/ee52f9e94e4b4c289a4a31c10d0905d4/users/f363a0851c4f46fb9322a5f14108b8a5/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:31.761110 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 796/1594] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:31.762092 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fc2e0-1ce1-471d-b0ed-0bc6f0d854cd admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:31.762402 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0026cf-bb69-44c8-b8e1-269da5c4b98b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee52f9e94e4b4c289a4a31c10d0905d4, user_id=f363a0851c4f46fb9322a5f14108b8a5, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:31.767560 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0026cf-bb69-44c8-b8e1-269da5c4b98b admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:31.773845 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fc2e0-1ce1-471d-b0ed-0bc6f0d854cd admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:31.778545 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 799/1595] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:31 2022] PUT /identity/v3/projects/ee52f9e94e4b4c289a4a31c10d0905d4/users/f363a0851c4f46fb9322a5f14108b8a5/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:31.790110 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216e353-849a-4c6f-833a-6eaea83c7964 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:31.790490 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216e353-849a-4c6f-833a-6eaea83c7964 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:31.790826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216e353-849a-4c6f-833a-6eaea83c7964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:31.819322 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-e216e353-849a-4c6f-833a-6eaea83c7964 None None] MFA Rules not processed for user `f363a0851c4f46fb9322a5f14108b8a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:31.820572 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e216e353-849a-4c6f-833a-6eaea83c7964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:31.842373 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d515f11-cce0-422a-96a3-f36f2657ccc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:31.843178 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d515f11-cce0-422a-96a3-f36f2657ccc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:31.843642 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d515f11-cce0-422a-96a3-f36f2657ccc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:31.843923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d515f11-cce0-422a-96a3-f36f2657ccc3 admin admin] PATH_INFO: `/v3/projects/095784c37a084530905c1d115805d267/users/35171261f21b44b2aa06bb6b38b1c46c/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:31.845860 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d515f11-cce0-422a-96a3-f36f2657ccc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=095784c37a084530905c1d115805d267, user_id=35171261f21b44b2aa06bb6b38b1c46c, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:31.848475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d515f11-cce0-422a-96a3-f36f2657ccc3 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:31.854968 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:31.860415 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 797/1596] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:31 2022] PUT /identity/v3/projects/095784c37a084530905c1d115805d267/users/35171261f21b44b2aa06bb6b38b1c46c/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:08:31.871709 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30d29b-3dfc-4ebe-ac9d-8b2cf592c841 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:31.872000 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30d29b-3dfc-4ebe-ac9d-8b2cf592c841 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:31.872287 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30d29b-3dfc-4ebe-ac9d-8b2cf592c841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:31.905118 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-9f30d29b-3dfc-4ebe-ac9d-8b2cf592c841 None None] MFA Rules not processed for user `35171261f21b44b2aa06bb6b38b1c46c`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:31.906677 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9f30d29b-3dfc-4ebe-ac9d-8b2cf592c841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:31.920060 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 800/1597] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:08:31.934709 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216e353-849a-4c6f-833a-6eaea83c7964 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:31.936108 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e216e353-849a-4c6f-833a-6eaea83c7964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:31.990814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 798/1598] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:31 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:32.004678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30d29b-3dfc-4ebe-ac9d-8b2cf592c841 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:32.006542 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9f30d29b-3dfc-4ebe-ac9d-8b2cf592c841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:32.088337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216e353-849a-4c6f-833a-6eaea83c7964 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:32.092384 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30d29b-3dfc-4ebe-ac9d-8b2cf592c841 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:32.096285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754057e-1c1f-4a4b-b859-b644aaea6b9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:32.097237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754057e-1c1f-4a4b-b859-b644aaea6b9b service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:32.097741 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754057e-1c1f-4a4b-b859-b644aaea6b9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:32.098242 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754057e-1c1f-4a4b-b859-b644aaea6b9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:32.102359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e93b4e-4976-4b66-898f-ef7aec36fc5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:32.102999 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e93b4e-4976-4b66-898f-ef7aec36fc5c service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:32.103327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e93b4e-4976-4b66-898f-ef7aec36fc5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:32.103609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e93b4e-4976-4b66-898f-ef7aec36fc5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:32.108696 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:32.116822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e93b4e-4976-4b66-898f-ef7aec36fc5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:32.118553 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e93b4e-4976-4b66-898f-ef7aec36fc5c service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:32.135781 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 799/1599] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:32 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:08:32.188734 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754057e-1c1f-4a4b-b859-b644aaea6b9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:32.190898 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754057e-1c1f-4a4b-b859-b644aaea6b9b service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:32.202919 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 801/1600] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:08:32.257395 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e93b4e-4976-4b66-898f-ef7aec36fc5c service cinder] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:32.272757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e93b4e-4976-4b66-898f-ef7aec36fc5c service cinder] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:32.273509 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e83706-82a4-4c4a-b13b-9fd6c868f824 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:32.274126 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e83706-82a4-4c4a-b13b-9fd6c868f824 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:32.274409 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e83706-82a4-4c4a-b13b-9fd6c868f824 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:32.274685 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e83706-82a4-4c4a-b13b-9fd6c868f824 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:32.305498 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754057e-1c1f-4a4b-b859-b644aaea6b9b service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:32.338138 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e83706-82a4-4c4a-b13b-9fd6c868f824 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:32.339585 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e83706-82a4-4c4a-b13b-9fd6c868f824 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:32.349567 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 800/1601] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:32.373123 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754057e-1c1f-4a4b-b859-b644aaea6b9b service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:32.378110 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829f226-50bc-413a-a219-0175269c8c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:32.378746 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829f226-50bc-413a-a219-0175269c8c91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:32.379060 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829f226-50bc-413a-a219-0175269c8c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:32.379370 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829f226-50bc-413a-a219-0175269c8c91 admin admin] PATH_INFO: `/v3/users/6d1031f083614c4c8847d1284d30977f` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:32.380527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829f226-50bc-413a-a219-0175269c8c91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d1031f083614c4c8847d1284d30977f)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:32.382434 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829f226-50bc-413a-a219-0175269c8c91 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:32.433668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c829f226-50bc-413a-a219-0175269c8c91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d1031f083614c4c8847d1284d30977f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:32.441633 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c829f226-50bc-413a-a219-0175269c8c91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d1031f083614c4c8847d1284d30977f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:32.454494 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c829f226-50bc-413a-a219-0175269c8c91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d1031f083614c4c8847d1284d30977f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:32.455191 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c829f226-50bc-413a-a219-0175269c8c91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d1031f083614c4c8847d1284d30977f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:32.455798 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c829f226-50bc-413a-a219-0175269c8c91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d1031f083614c4c8847d1284d30977f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:32.469609 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 802/1602] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:32 2022] DELETE /identity/v3/users/6d1031f083614c4c8847d1284d30977f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:08:32.474701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777bbdfe-f642-4ee5-adb8-69e1a428c0e5 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:32.475002 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777bbdfe-f642-4ee5-adb8-69e1a428c0e5 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:32.476015 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777bbdfe-f642-4ee5-adb8-69e1a428c0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:32.495989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-777bbdfe-f642-4ee5-adb8-69e1a428c0e5 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:32.497234 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-777bbdfe-f642-4ee5-adb8-69e1a428c0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:32.518095 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:32.563781 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 801/1603] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:32.574693 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829f226-50bc-413a-a219-0175269c8c91 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:32.576172 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-c829f226-50bc-413a-a219-0175269c8c91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:32.611157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:32.668967 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829f226-50bc-413a-a219-0175269c8c91 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:32.682945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c54a1e-417c-4539-924f-dc96d0982baa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:32.684115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c54a1e-417c-4539-924f-dc96d0982baa service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:32.684658 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c54a1e-417c-4539-924f-dc96d0982baa service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:32.685084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c54a1e-417c-4539-924f-dc96d0982baa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:32.699900 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c54a1e-417c-4539-924f-dc96d0982baa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:32.701588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c54a1e-417c-4539-924f-dc96d0982baa service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:32.727329 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 803/1604] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:08:33.081677 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777bbdfe-f642-4ee5-adb8-69e1a428c0e5 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:33.083170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-777bbdfe-f642-4ee5-adb8-69e1a428c0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:33.096211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777bbdfe-f642-4ee5-adb8-69e1a428c0e5 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:33.101045 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8925dac-ada1-47ca-b086-ee9695fd5783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:33.101695 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8925dac-ada1-47ca-b086-ee9695fd5783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:33.101982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8925dac-ada1-47ca-b086-ee9695fd5783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:33.102249 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8925dac-ada1-47ca-b086-ee9695fd5783 admin admin] PATH_INFO: `/v3/projects/38e129eab49f4b34973e8b7da5c36bfc` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:33.104332 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8925dac-ada1-47ca-b086-ee9695fd5783 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38e129eab49f4b34973e8b7da5c36bfc)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:33.106312 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8925dac-ada1-47ca-b086-ee9695fd5783 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:33.154641 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-c8925dac-ada1-47ca-b086-ee9695fd5783 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38e129eab49f4b34973e8b7da5c36bfc', 'request_id': 'req-c8925dac-ada1-47ca-b086-ee9695fd5783'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:33.161823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-c8925dac-ada1-47ca-b086-ee9695fd5783 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38e129eab49f4b34973e8b7da5c36bfc', 'request_id': 'req-c8925dac-ada1-47ca-b086-ee9695fd5783'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:33.167308 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-c8925dac-ada1-47ca-b086-ee9695fd5783 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38e129eab49f4b34973e8b7da5c36bfc', 'request_id': 'req-c8925dac-ada1-47ca-b086-ee9695fd5783'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:33.178962 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-c8925dac-ada1-47ca-b086-ee9695fd5783 admin admin] The token cache is being invalidate because project 38e129eab49f4b34973e8b7da5c36bfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:33.179249 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-c8925dac-ada1-47ca-b086-ee9695fd5783 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:33.179875 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-c8925dac-ada1-47ca-b086-ee9695fd5783 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:33.181384 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 802/1605] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:33 2022] DELETE /identity/v3/projects/38e129eab49f4b34973e8b7da5c36bfc => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:33.185472 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c54a1e-417c-4539-924f-dc96d0982baa service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:33.196079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c54a1e-417c-4539-924f-dc96d0982baa service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:33.252649 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19855ac4-9cd1-44c5-9fc7-d3915fee9582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:33.253201 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19855ac4-9cd1-44c5-9fc7-d3915fee9582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:33.253483 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19855ac4-9cd1-44c5-9fc7-d3915fee9582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:33.253756 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19855ac4-9cd1-44c5-9fc7-d3915fee9582 admin admin] PATH_INFO: `/v3/users/35171261f21b44b2aa06bb6b38b1c46c` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:33.254867 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19855ac4-9cd1-44c5-9fc7-d3915fee9582 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35171261f21b44b2aa06bb6b38b1c46c)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:33.256789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19855ac4-9cd1-44c5-9fc7-d3915fee9582 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:33.305554 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-19855ac4-9cd1-44c5-9fc7-d3915fee9582 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35171261f21b44b2aa06bb6b38b1c46c'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:33.313289 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-19855ac4-9cd1-44c5-9fc7-d3915fee9582 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35171261f21b44b2aa06bb6b38b1c46c'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:33.325951 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-19855ac4-9cd1-44c5-9fc7-d3915fee9582 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35171261f21b44b2aa06bb6b38b1c46c'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:33.326677 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-19855ac4-9cd1-44c5-9fc7-d3915fee9582 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35171261f21b44b2aa06bb6b38b1c46c'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:33.327437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-19855ac4-9cd1-44c5-9fc7-d3915fee9582 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35171261f21b44b2aa06bb6b38b1c46c'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:33.345122 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 804/1606] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:33 2022] DELETE /identity/v3/users/35171261f21b44b2aa06bb6b38b1c46c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:33.796477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8925dac-ada1-47ca-b086-ee9695fd5783 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:33.809322 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c8925dac-ada1-47ca-b086-ee9695fd5783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:33.821765 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8925dac-ada1-47ca-b086-ee9695fd5783 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:33.883340 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21881747-4e95-493b-b2eb-5bfa48c9067f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:33.884021 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21881747-4e95-493b-b2eb-5bfa48c9067f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:33.884311 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21881747-4e95-493b-b2eb-5bfa48c9067f admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:33.884630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21881747-4e95-493b-b2eb-5bfa48c9067f admin admin] PATH_INFO: `/v3/projects/095784c37a084530905c1d115805d267` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:33.885802 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21881747-4e95-493b-b2eb-5bfa48c9067f admin admin] RBAC: Authorizing `identity:delete_project(project_id=095784c37a084530905c1d115805d267)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:33.888440 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21881747-4e95-493b-b2eb-5bfa48c9067f admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:33.935366 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-21881747-4e95-493b-b2eb-5bfa48c9067f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '095784c37a084530905c1d115805d267', 'request_id': 'req-21881747-4e95-493b-b2eb-5bfa48c9067f'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:33.938139 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:33.943330 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-21881747-4e95-493b-b2eb-5bfa48c9067f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '095784c37a084530905c1d115805d267', 'request_id': 'req-21881747-4e95-493b-b2eb-5bfa48c9067f'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:33.950645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-21881747-4e95-493b-b2eb-5bfa48c9067f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '095784c37a084530905c1d115805d267', 'request_id': 'req-21881747-4e95-493b-b2eb-5bfa48c9067f'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:33.962018 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-21881747-4e95-493b-b2eb-5bfa48c9067f admin admin] The token cache is being invalidate because project 095784c37a084530905c1d115805d267 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:33.962345 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-21881747-4e95-493b-b2eb-5bfa48c9067f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:33.962974 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-21881747-4e95-493b-b2eb-5bfa48c9067f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:33.964443 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 803/1607] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:33 2022] DELETE /identity/v3/projects/095784c37a084530905c1d115805d267 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:33.970120 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0b210-cfc5-4fa5-bd1c-6b2451a273d1 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:33.970512 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0b210-cfc5-4fa5-bd1c-6b2451a273d1 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:33.970860 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0b210-cfc5-4fa5-bd1c-6b2451a273d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:33.991286 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1af0b210-cfc5-4fa5-bd1c-6b2451a273d1 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:33.992598 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1af0b210-cfc5-4fa5-bd1c-6b2451a273d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:34.040601 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:34.062962 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 805/1608] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:34.070739 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91165c66-1178-41cf-9351-a5790ae11a42 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:34.071032 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91165c66-1178-41cf-9351-a5790ae11a42 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:34.071307 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91165c66-1178-41cf-9351-a5790ae11a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:34.092257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-91165c66-1178-41cf-9351-a5790ae11a42 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:34.094238 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-91165c66-1178-41cf-9351-a5790ae11a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:34.102720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 804/1609] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:34.107279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0b210-cfc5-4fa5-bd1c-6b2451a273d1 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:34.120400 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0b210-cfc5-4fa5-bd1c-6b2451a273d1 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:34.126836 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710df8c9-3fca-438b-8af8-da19bb6b307f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:34.127448 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710df8c9-3fca-438b-8af8-da19bb6b307f admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:34.127730 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710df8c9-3fca-438b-8af8-da19bb6b307f admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:34.128001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710df8c9-3fca-438b-8af8-da19bb6b307f admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:34.128690 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710df8c9-3fca-438b-8af8-da19bb6b307f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:08:34.128998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710df8c9-3fca-438b-8af8-da19bb6b307f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:34.130583 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710df8c9-3fca-438b-8af8-da19bb6b307f admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:34.143181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 806/1610] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:08:34.147450 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91165c66-1178-41cf-9351-a5790ae11a42 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:34.157643 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91165c66-1178-41cf-9351-a5790ae11a42 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:34.162361 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a443785-74d5-4964-8738-4d459115041f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:34.162969 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a443785-74d5-4964-8738-4d459115041f admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:34.163255 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a443785-74d5-4964-8738-4d459115041f admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:34.163569 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a443785-74d5-4964-8738-4d459115041f admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:34.164153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a443785-74d5-4964-8738-4d459115041f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:34.166076 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a443785-74d5-4964-8738-4d459115041f admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:34.194243 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 805/1611] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:34 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:34.198372 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710df8c9-3fca-438b-8af8-da19bb6b307f admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:34.208316 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710df8c9-3fca-438b-8af8-da19bb6b307f admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:34.265808 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1936fe6-6bc8-4ce8-8200-2a2dab21697f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:34.266406 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1936fe6-6bc8-4ce8-8200-2a2dab21697f admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:34.266689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1936fe6-6bc8-4ce8-8200-2a2dab21697f admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:34.266966 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1936fe6-6bc8-4ce8-8200-2a2dab21697f admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:34.267595 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1936fe6-6bc8-4ce8-8200-2a2dab21697f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:34.269528 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1936fe6-6bc8-4ce8-8200-2a2dab21697f admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:34.293671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 807/1612] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:34 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:34.294907 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a443785-74d5-4964-8738-4d459115041f admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:34.305680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a443785-74d5-4964-8738-4d459115041f admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:34.310308 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ef3ba-0e53-4be0-8379-e4eae29e9f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:34.310901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ef3ba-0e53-4be0-8379-e4eae29e9f2d admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:34.311183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ef3ba-0e53-4be0-8379-e4eae29e9f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:34.311463 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1ef3ba-0e53-4be0-8379-e4eae29e9f2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:34.312100 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ef3ba-0e53-4be0-8379-e4eae29e9f2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:34.313784 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1ef3ba-0e53-4be0-8379-e4eae29e9f2d admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:34.323691 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 806/1613] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:34.328180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1936fe6-6bc8-4ce8-8200-2a2dab21697f admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:34.338197 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1936fe6-6bc8-4ce8-8200-2a2dab21697f admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:34.343845 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8ec7b-49bb-4648-b1d5-6f6ba5e20e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:34.344399 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8ec7b-49bb-4648-b1d5-6f6ba5e20e99 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:34.344734 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8ec7b-49bb-4648-b1d5-6f6ba5e20e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:34.345032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8ec7b-49bb-4648-b1d5-6f6ba5e20e99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:34.345674 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8ec7b-49bb-4648-b1d5-6f6ba5e20e99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:34.347241 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8ec7b-49bb-4648-b1d5-6f6ba5e20e99 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:34.358415 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 808/1614] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:34.363202 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ef3ba-0e53-4be0-8379-e4eae29e9f2d admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:34.373227 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1ef3ba-0e53-4be0-8379-e4eae29e9f2d admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:34.378338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152636e8-12ce-49b4-8532-e162d52df8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:34.379001 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152636e8-12ce-49b4-8532-e162d52df8bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:34.379306 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152636e8-12ce-49b4-8532-e162d52df8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:34.379579 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152636e8-12ce-49b4-8532-e162d52df8bf admin admin] PATH_INFO: `/v3/projects/f8240da88b5d4ed497a84117649085e9/users/8851678eb2684a78b013903b87e6ed6b/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:34.396243 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152636e8-12ce-49b4-8532-e162d52df8bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8240da88b5d4ed497a84117649085e9, user_id=8851678eb2684a78b013903b87e6ed6b, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:34.398984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152636e8-12ce-49b4-8532-e162d52df8bf admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:34.412233 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 807/1615] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:34 2022] PUT /identity/v3/projects/f8240da88b5d4ed497a84117649085e9/users/8851678eb2684a78b013903b87e6ed6b/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:34.425784 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934aacba-b79d-4633-b77b-6168798914ff None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:34.426077 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934aacba-b79d-4633-b77b-6168798914ff None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:34.426354 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934aacba-b79d-4633-b77b-6168798914ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:34.455377 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-934aacba-b79d-4633-b77b-6168798914ff None None] MFA Rules not processed for user `8851678eb2684a78b013903b87e6ed6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:34.456608 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-934aacba-b79d-4633-b77b-6168798914ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:34.523760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 809/1616] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:34.528579 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152636e8-12ce-49b4-8532-e162d52df8bf admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:34.595111 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152636e8-12ce-49b4-8532-e162d52df8bf admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:34.600330 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ca902-bd56-4c37-a565-deae9d79e69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:34.600993 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ca902-bd56-4c37-a565-deae9d79e69c admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:34.601457 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ca902-bd56-4c37-a565-deae9d79e69c admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:34.601990 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976ca902-bd56-4c37-a565-deae9d79e69c admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:34.602745 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ca902-bd56-4c37-a565-deae9d79e69c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:34.604691 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976ca902-bd56-4c37-a565-deae9d79e69c admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:34.638942 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 808/1617] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:34 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:34.644060 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934aacba-b79d-4633-b77b-6168798914ff None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:34.654967 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934aacba-b79d-4633-b77b-6168798914ff None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:34.696765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:34.719486 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185cf857-114f-4f5f-bed7-14b5e0c6d1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:34.720175 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185cf857-114f-4f5f-bed7-14b5e0c6d1f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:34.720560 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185cf857-114f-4f5f-bed7-14b5e0c6d1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:34.720908 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185cf857-114f-4f5f-bed7-14b5e0c6d1f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:34.721645 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185cf857-114f-4f5f-bed7-14b5e0c6d1f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:34.723632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185cf857-114f-4f5f-bed7-14b5e0c6d1f4 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:34.750708 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 810/1618] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:34 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:34.755649 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ca902-bd56-4c37-a565-deae9d79e69c admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:34.768022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976ca902-bd56-4c37-a565-deae9d79e69c admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:34.773472 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4a120-caef-4398-a9fe-7d99388c104e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:34.773864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4a120-caef-4398-a9fe-7d99388c104e admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:34.774221 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4a120-caef-4398-a9fe-7d99388c104e admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:34.774694 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4a120-caef-4398-a9fe-7d99388c104e admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:34.775409 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4a120-caef-4398-a9fe-7d99388c104e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:34.777123 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4a120-caef-4398-a9fe-7d99388c104e admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:34.791814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 809/1619] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:34.793503 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185cf857-114f-4f5f-bed7-14b5e0c6d1f4 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:34.806773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185cf857-114f-4f5f-bed7-14b5e0c6d1f4 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:34.812709 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65dadcc-7ca9-428d-96b6-4c5c68f88b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:34.813609 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65dadcc-7ca9-428d-96b6-4c5c68f88b2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:34.813958 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65dadcc-7ca9-428d-96b6-4c5c68f88b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:34.815764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65dadcc-7ca9-428d-96b6-4c5c68f88b2a admin admin] PATH_INFO: `/v3/projects/24821bd6a9ea45958e4c60d56e94689e/users/007f42224bfc4ccb9e94735c385d7430/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:34.837212 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65dadcc-7ca9-428d-96b6-4c5c68f88b2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=24821bd6a9ea45958e4c60d56e94689e, user_id=007f42224bfc4ccb9e94735c385d7430, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:34.839948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65dadcc-7ca9-428d-96b6-4c5c68f88b2a admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:34.855636 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 811/1620] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:34 2022] PUT /identity/v3/projects/24821bd6a9ea45958e4c60d56e94689e/users/007f42224bfc4ccb9e94735c385d7430/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:34.865990 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4a120-caef-4398-a9fe-7d99388c104e admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:34.878581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4a120-caef-4398-a9fe-7d99388c104e admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:34.949588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:34.962380 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d01dd-feda-4744-b9ae-fc4d969ebc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:34.963049 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d01dd-feda-4744-b9ae-fc4d969ebc45 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:34.963334 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d01dd-feda-4744-b9ae-fc4d969ebc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:34.963611 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d01dd-feda-4744-b9ae-fc4d969ebc45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:34.964247 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d01dd-feda-4744-b9ae-fc4d969ebc45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:34.965895 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d01dd-feda-4744-b9ae-fc4d969ebc45 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:34.976274 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 810/1621] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:34.980701 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65dadcc-7ca9-428d-96b6-4c5c68f88b2a admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:34.994411 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65dadcc-7ca9-428d-96b6-4c5c68f88b2a admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:35.000685 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075ca7a-21a7-49e6-b84a-dab7c397bd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:35.001392 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075ca7a-21a7-49e6-b84a-dab7c397bd74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:35.001728 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075ca7a-21a7-49e6-b84a-dab7c397bd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:35.002074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075ca7a-21a7-49e6-b84a-dab7c397bd74 admin admin] PATH_INFO: `/v3/projects/24821bd6a9ea45958e4c60d56e94689e/users/007f42224bfc4ccb9e94735c385d7430/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:35.004282 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075ca7a-21a7-49e6-b84a-dab7c397bd74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24821bd6a9ea45958e4c60d56e94689e, user_id=007f42224bfc4ccb9e94735c385d7430, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:35.007309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0075ca7a-21a7-49e6-b84a-dab7c397bd74 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:35.018664 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 812/1622] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:34 2022] PUT /identity/v3/projects/24821bd6a9ea45958e4c60d56e94689e/users/007f42224bfc4ccb9e94735c385d7430/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:08:35.029680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2373f2-728e-420c-b384-61a62d138b9c None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:35.030160 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2373f2-728e-420c-b384-61a62d138b9c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:35.030658 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2373f2-728e-420c-b384-61a62d138b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:35.069932 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-0a2373f2-728e-420c-b384-61a62d138b9c None None] MFA Rules not processed for user `007f42224bfc4ccb9e94735c385d7430`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:35.071734 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-0a2373f2-728e-420c-b384-61a62d138b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:35.150633 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 811/1623] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:35 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:35.214969 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075ca7a-21a7-49e6-b84a-dab7c397bd74 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:35.216735 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-0075ca7a-21a7-49e6-b84a-dab7c397bd74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:35.315373 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075ca7a-21a7-49e6-b84a-dab7c397bd74 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:35.319291 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454d93e-bd56-4370-a2c6-ebae9d6dc2de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:35.320040 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454d93e-bd56-4370-a2c6-ebae9d6dc2de service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:35.320429 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454d93e-bd56-4370-a2c6-ebae9d6dc2de service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:35.320841 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454d93e-bd56-4370-a2c6-ebae9d6dc2de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:35.331832 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0454d93e-bd56-4370-a2c6-ebae9d6dc2de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:35.333468 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0454d93e-bd56-4370-a2c6-ebae9d6dc2de service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:35.344596 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 813/1624] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:35.763049 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2373f2-728e-420c-b384-61a62d138b9c None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:35.824076 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2373f2-728e-420c-b384-61a62d138b9c None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:35.829251 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ef6b2-495e-410c-971f-da679b8e6a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:35.829878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ef6b2-495e-410c-971f-da679b8e6a74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:35.830159 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ef6b2-495e-410c-971f-da679b8e6a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:35.830430 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ef6b2-495e-410c-971f-da679b8e6a74 admin admin] PATH_INFO: `/v3/users/8851678eb2684a78b013903b87e6ed6b` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:35.831552 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ef6b2-495e-410c-971f-da679b8e6a74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8851678eb2684a78b013903b87e6ed6b)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:35.833481 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ef6b2-495e-410c-971f-da679b8e6a74 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:35.885520 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-173ef6b2-495e-410c-971f-da679b8e6a74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8851678eb2684a78b013903b87e6ed6b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:35.886150 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-173ef6b2-495e-410c-971f-da679b8e6a74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8851678eb2684a78b013903b87e6ed6b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:35.895377 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-173ef6b2-495e-410c-971f-da679b8e6a74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8851678eb2684a78b013903b87e6ed6b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:35.907878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-173ef6b2-495e-410c-971f-da679b8e6a74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8851678eb2684a78b013903b87e6ed6b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:35.910449 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:35.919254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-173ef6b2-495e-410c-971f-da679b8e6a74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8851678eb2684a78b013903b87e6ed6b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:35.927742 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 812/1625] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:35 2022] DELETE /identity/v3/users/8851678eb2684a78b013903b87e6ed6b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:35.932228 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba174c7-8320-42c0-bf47-3ad4765cf4b8 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:35.932891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba174c7-8320-42c0-bf47-3ad4765cf4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:35.933609 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba174c7-8320-42c0-bf47-3ad4765cf4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:35.974711 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-aba174c7-8320-42c0-bf47-3ad4765cf4b8 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:35.976514 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-aba174c7-8320-42c0-bf47-3ad4765cf4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:36.062884 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 814/1626] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:36.072850 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ef6b2-495e-410c-971f-da679b8e6a74 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:36.075791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-173ef6b2-495e-410c-971f-da679b8e6a74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:36.183837 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ef6b2-495e-410c-971f-da679b8e6a74 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:36.188427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78f617-24be-4c4d-9069-9725fb8a5db6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:36.189243 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78f617-24be-4c4d-9069-9725fb8a5db6 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:36.189696 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78f617-24be-4c4d-9069-9725fb8a5db6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:36.190124 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78f617-24be-4c4d-9069-9725fb8a5db6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:36.201465 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78f617-24be-4c4d-9069-9725fb8a5db6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:36.212155 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78f617-24be-4c4d-9069-9725fb8a5db6 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:36.225393 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 813/1627] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:08:36.664525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba174c7-8320-42c0-bf47-3ad4765cf4b8 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:36.666021 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-aba174c7-8320-42c0-bf47-3ad4765cf4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:36.678569 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba174c7-8320-42c0-bf47-3ad4765cf4b8 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:36.684535 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a796ea3-69ec-4334-86c9-ee251a8be0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:36.685222 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a796ea3-69ec-4334-86c9-ee251a8be0cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:36.685445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a796ea3-69ec-4334-86c9-ee251a8be0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:36.685719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a796ea3-69ec-4334-86c9-ee251a8be0cf admin admin] PATH_INFO: `/v3/projects/f8240da88b5d4ed497a84117649085e9` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:36.686864 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a796ea3-69ec-4334-86c9-ee251a8be0cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8240da88b5d4ed497a84117649085e9)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:36.688799 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a796ea3-69ec-4334-86c9-ee251a8be0cf admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:36.723376 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:36.739121 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1a796ea3-69ec-4334-86c9-ee251a8be0cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8240da88b5d4ed497a84117649085e9', 'request_id': 'req-1a796ea3-69ec-4334-86c9-ee251a8be0cf'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:36.745361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1a796ea3-69ec-4334-86c9-ee251a8be0cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8240da88b5d4ed497a84117649085e9', 'request_id': 'req-1a796ea3-69ec-4334-86c9-ee251a8be0cf'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:36.753053 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1a796ea3-69ec-4334-86c9-ee251a8be0cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8240da88b5d4ed497a84117649085e9', 'request_id': 'req-1a796ea3-69ec-4334-86c9-ee251a8be0cf'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:36.763895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1a796ea3-69ec-4334-86c9-ee251a8be0cf admin admin] The token cache is being invalidate because project f8240da88b5d4ed497a84117649085e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:36.764184 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1a796ea3-69ec-4334-86c9-ee251a8be0cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:36.765713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1a796ea3-69ec-4334-86c9-ee251a8be0cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:36.767155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 815/1628] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:36 2022] DELETE /identity/v3/projects/f8240da88b5d4ed497a84117649085e9 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:36.771281 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78f617-24be-4c4d-9069-9725fb8a5db6 service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:36.781872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78f617-24be-4c4d-9069-9725fb8a5db6 service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:36.838329 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf1004-314a-42e1-ad42-b6e1f8bf5c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:36.838958 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf1004-314a-42e1-ad42-b6e1f8bf5c08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:36.839242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf1004-314a-42e1-ad42-b6e1f8bf5c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:36.839518 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf1004-314a-42e1-ad42-b6e1f8bf5c08 admin admin] PATH_INFO: `/v3/users/007f42224bfc4ccb9e94735c385d7430` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:36.840667 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf1004-314a-42e1-ad42-b6e1f8bf5c08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=007f42224bfc4ccb9e94735c385d7430)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:36.843476 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf1004-314a-42e1-ad42-b6e1f8bf5c08 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:36.887660 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-dcbf1004-314a-42e1-ad42-b6e1f8bf5c08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '007f42224bfc4ccb9e94735c385d7430'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:36.888307 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-dcbf1004-314a-42e1-ad42-b6e1f8bf5c08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '007f42224bfc4ccb9e94735c385d7430'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:36.895885 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-dcbf1004-314a-42e1-ad42-b6e1f8bf5c08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '007f42224bfc4ccb9e94735c385d7430'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:36.908473 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-dcbf1004-314a-42e1-ad42-b6e1f8bf5c08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '007f42224bfc4ccb9e94735c385d7430'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:36.915919 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-dcbf1004-314a-42e1-ad42-b6e1f8bf5c08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '007f42224bfc4ccb9e94735c385d7430'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:36.922105 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 814/1629] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:36 2022] DELETE /identity/v3/users/007f42224bfc4ccb9e94735c385d7430 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:37.214002 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a796ea3-69ec-4334-86c9-ee251a8be0cf admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:37.215771 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1a796ea3-69ec-4334-86c9-ee251a8be0cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:37.232128 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a796ea3-69ec-4334-86c9-ee251a8be0cf admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:37.317582 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37844b-0a07-4e77-b7a1-3a8200f900a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:37.318247 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37844b-0a07-4e77-b7a1-3a8200f900a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:37.318535 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37844b-0a07-4e77-b7a1-3a8200f900a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:37.318809 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37844b-0a07-4e77-b7a1-3a8200f900a1 admin admin] PATH_INFO: `/v3/projects/24821bd6a9ea45958e4c60d56e94689e` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:37.320223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37844b-0a07-4e77-b7a1-3a8200f900a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24821bd6a9ea45958e4c60d56e94689e)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:37.322244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37844b-0a07-4e77-b7a1-3a8200f900a1 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:37.385175 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-cb37844b-0a07-4e77-b7a1-3a8200f900a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24821bd6a9ea45958e4c60d56e94689e', 'request_id': 'req-cb37844b-0a07-4e77-b7a1-3a8200f900a1'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:37.393835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-cb37844b-0a07-4e77-b7a1-3a8200f900a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24821bd6a9ea45958e4c60d56e94689e', 'request_id': 'req-cb37844b-0a07-4e77-b7a1-3a8200f900a1'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:37.418847 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-cb37844b-0a07-4e77-b7a1-3a8200f900a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24821bd6a9ea45958e4c60d56e94689e', 'request_id': 'req-cb37844b-0a07-4e77-b7a1-3a8200f900a1'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:37.435593 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-cb37844b-0a07-4e77-b7a1-3a8200f900a1 admin admin] The token cache is being invalidate because project 24821bd6a9ea45958e4c60d56e94689e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:37.435967 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-cb37844b-0a07-4e77-b7a1-3a8200f900a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:37.436950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-cb37844b-0a07-4e77-b7a1-3a8200f900a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:37.438600 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 816/1630] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:37 2022] DELETE /identity/v3/projects/24821bd6a9ea45958e4c60d56e94689e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:37.443881 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bbc22-535d-4135-adee-e26b83bd2a75 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:37.444183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bbc22-535d-4135-adee-e26b83bd2a75 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:37.444465 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bbc22-535d-4135-adee-e26b83bd2a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:37.469632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-4b9bbc22-535d-4135-adee-e26b83bd2a75 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:37.471053 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-4b9bbc22-535d-4135-adee-e26b83bd2a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:37.480706 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:37.549837 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 815/1631] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:37.553220 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f6a14-0d92-41c1-a6f9-5b65b9f5aedf None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:37.553839 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f6a14-0d92-41c1-a6f9-5b65b9f5aedf None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:37.554193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f6a14-0d92-41c1-a6f9-5b65b9f5aedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:37.579591 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-fd5f6a14-0d92-41c1-a6f9-5b65b9f5aedf None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:37.580871 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-fd5f6a14-0d92-41c1-a6f9-5b65b9f5aedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:37.590181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 817/1632] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:37.593841 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9bbc22-535d-4135-adee-e26b83bd2a75 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:37.604241 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9bbc22-535d-4135-adee-e26b83bd2a75 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:37.609378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e4c9e-81fb-4e8d-9a44-c9523d99b746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:37.609965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e4c9e-81fb-4e8d-9a44-c9523d99b746 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:37.610242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e4c9e-81fb-4e8d-9a44-c9523d99b746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:37.610516 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e4c9e-81fb-4e8d-9a44-c9523d99b746 admin admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:37.611155 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e4c9e-81fb-4e8d-9a44-c9523d99b746 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:08:37.611454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e4c9e-81fb-4e8d-9a44-c9523d99b746 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:37.613049 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e4c9e-81fb-4e8d-9a44-c9523d99b746 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:37.627596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 816/1633] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:08:37.629571 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f6a14-0d92-41c1-a6f9-5b65b9f5aedf None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:37.638752 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f6a14-0d92-41c1-a6f9-5b65b9f5aedf None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:37.643501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7423a-9367-4b83-8dc5-8a75fdd6ade3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:37.644091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7423a-9367-4b83-8dc5-8a75fdd6ade3 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:37.644367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7423a-9367-4b83-8dc5-8a75fdd6ade3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:37.644673 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7423a-9367-4b83-8dc5-8a75fdd6ade3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:37.645325 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7423a-9367-4b83-8dc5-8a75fdd6ade3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:37.647177 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7423a-9367-4b83-8dc5-8a75fdd6ade3 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:37.677291 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 818/1634] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:37 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:08:37.681355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e4c9e-81fb-4e8d-9a44-c9523d99b746 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:37.692388 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e4c9e-81fb-4e8d-9a44-c9523d99b746 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:37.754235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb54cce-6feb-4021-b588-38af4aead515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:37.754849 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb54cce-6feb-4021-b588-38af4aead515 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:37.755130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb54cce-6feb-4021-b588-38af4aead515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:37.755398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb54cce-6feb-4021-b588-38af4aead515 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:37.756061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb54cce-6feb-4021-b588-38af4aead515 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:37.758051 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb54cce-6feb-4021-b588-38af4aead515 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:37.784624 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 817/1635] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:37 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:08:37.786365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7423a-9367-4b83-8dc5-8a75fdd6ade3 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:37.796569 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7423a-9367-4b83-8dc5-8a75fdd6ade3 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:37.801388 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662395f2-914d-475c-95ab-d7f7e91b75ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:37.801999 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662395f2-914d-475c-95ab-d7f7e91b75ea admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:37.802287 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662395f2-914d-475c-95ab-d7f7e91b75ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:37.802569 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662395f2-914d-475c-95ab-d7f7e91b75ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:37.803211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662395f2-914d-475c-95ab-d7f7e91b75ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:37.804860 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662395f2-914d-475c-95ab-d7f7e91b75ea admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:37.817405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 819/1636] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:37.821589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb54cce-6feb-4021-b588-38af4aead515 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:37.832165 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb54cce-6feb-4021-b588-38af4aead515 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:37.840441 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc5007-efd3-4dd0-bc37-c26d64e419a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:37.841211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc5007-efd3-4dd0-bc37-c26d64e419a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:37.841640 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc5007-efd3-4dd0-bc37-c26d64e419a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:37.841926 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc5007-efd3-4dd0-bc37-c26d64e419a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:37.842870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc5007-efd3-4dd0-bc37-c26d64e419a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:37.844630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc5007-efd3-4dd0-bc37-c26d64e419a9 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:37.855756 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 818/1637] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:08:37.860350 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662395f2-914d-475c-95ab-d7f7e91b75ea admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:37.870637 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662395f2-914d-475c-95ab-d7f7e91b75ea admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:37.876175 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb0686-7a39-4882-9cc5-5a167f5a8b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:37.876893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb0686-7a39-4882-9cc5-5a167f5a8b75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:37.877187 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb0686-7a39-4882-9cc5-5a167f5a8b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:37.877468 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb0686-7a39-4882-9cc5-5a167f5a8b75 admin admin] PATH_INFO: `/v3/projects/56497cbfe627422fb354905e112446c2/users/e343e4b895854d2e852afae3953de172/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:37.880670 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:37.895423 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb0686-7a39-4882-9cc5-5a167f5a8b75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56497cbfe627422fb354905e112446c2, user_id=e343e4b895854d2e852afae3953de172, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:37.898254 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb0686-7a39-4882-9cc5-5a167f5a8b75 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:37.911192 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 820/1638] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:37 2022] PUT /identity/v3/projects/56497cbfe627422fb354905e112446c2/users/e343e4b895854d2e852afae3953de172/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:08:37.922741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f6e8d-8c80-4be1-b55a-7f70f996b2fc None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:37.923297 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f6e8d-8c80-4be1-b55a-7f70f996b2fc None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:37.923297 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f6e8d-8c80-4be1-b55a-7f70f996b2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:37.952900 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-ae0f6e8d-8c80-4be1-b55a-7f70f996b2fc None None] MFA Rules not processed for user `e343e4b895854d2e852afae3953de172`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:37.954507 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ae0f6e8d-8c80-4be1-b55a-7f70f996b2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:38.023451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 819/1639] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:38.034523 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb0686-7a39-4882-9cc5-5a167f5a8b75 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:38.036217 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-56eb0686-7a39-4882-9cc5-5a167f5a8b75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:38.118969 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb0686-7a39-4882-9cc5-5a167f5a8b75 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:38.122608 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29b31c-a5c4-4e39-8240-c6a88375778c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a69aeb3fba49b6881f1f15c1758494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:38.123216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29b31c-a5c4-4e39-8240-c6a88375778c service glance] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:38.123509 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29b31c-a5c4-4e39-8240-c6a88375778c service glance] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:38.123777 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29b31c-a5c4-4e39-8240-c6a88375778c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:38.135159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e29b31c-a5c4-4e39-8240-c6a88375778c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:38.136609 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e29b31c-a5c4-4e39-8240-c6a88375778c service glance] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:38.145485 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 821/1640] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:40.591580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f6e8d-8c80-4be1-b55a-7f70f996b2fc None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:40.593296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ae0f6e8d-8c80-4be1-b55a-7f70f996b2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:40.604202 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:40.655948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f6e8d-8c80-4be1-b55a-7f70f996b2fc None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:40.659733 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c0d59-ae51-4a19-b46e-cb129c513ba1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:40.660350 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c0d59-ae51-4a19-b46e-cb129c513ba1 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:40.660761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c0d59-ae51-4a19-b46e-cb129c513ba1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:40.661022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c0d59-ae51-4a19-b46e-cb129c513ba1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:40.662780 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-639c0d59-ae51-4a19-b46e-cb129c513ba1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:40.742418 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c0d59-ae51-4a19-b46e-cb129c513ba1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:40.743962 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c0d59-ae51-4a19-b46e-cb129c513ba1 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:40.756961 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 820/1641] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:40.856664 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29b31c-a5c4-4e39-8240-c6a88375778c service glance] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:40.859734 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1e29b31c-a5c4-4e39-8240-c6a88375778c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:40.974861 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29b31c-a5c4-4e39-8240-c6a88375778c service glance] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:40.980066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ade86-dbaa-4efd-9e68-31f05f1516f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:40.980947 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ade86-dbaa-4efd-9e68-31f05f1516f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:40.981304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ade86-dbaa-4efd-9e68-31f05f1516f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:40.981657 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ade86-dbaa-4efd-9e68-31f05f1516f3 admin admin] PATH_INFO: `/v3/users/4d187ce9dfb648a9abc51e3ee3f464b6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:40.983021 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ade86-dbaa-4efd-9e68-31f05f1516f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d187ce9dfb648a9abc51e3ee3f464b6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:40.985503 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ade86-dbaa-4efd-9e68-31f05f1516f3 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:41.100002 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-114ade86-dbaa-4efd-9e68-31f05f1516f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d187ce9dfb648a9abc51e3ee3f464b6'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:41.129539 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-114ade86-dbaa-4efd-9e68-31f05f1516f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d187ce9dfb648a9abc51e3ee3f464b6'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:41.158945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:41.165795 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-114ade86-dbaa-4efd-9e68-31f05f1516f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d187ce9dfb648a9abc51e3ee3f464b6'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:41.167084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-114ade86-dbaa-4efd-9e68-31f05f1516f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d187ce9dfb648a9abc51e3ee3f464b6'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:41.167735 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-114ade86-dbaa-4efd-9e68-31f05f1516f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d187ce9dfb648a9abc51e3ee3f464b6'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:41.188606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 822/1642] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:40 2022] DELETE /identity/v3/users/4d187ce9dfb648a9abc51e3ee3f464b6 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:41.199843 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb5ac7-0010-45c9-9e72-6b47cc8fd45b None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:41.199843 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb5ac7-0010-45c9-9e72-6b47cc8fd45b None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:41.199843 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb5ac7-0010-45c9-9e72-6b47cc8fd45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:41.221876 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-aceb5ac7-0010-45c9-9e72-6b47cc8fd45b None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:41.223676 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-aceb5ac7-0010-45c9-9e72-6b47cc8fd45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:41.396260 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 821/1643] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:41.405476 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ade86-dbaa-4efd-9e68-31f05f1516f3 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:41.407046 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-114ade86-dbaa-4efd-9e68-31f05f1516f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:41.566759 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ade86-dbaa-4efd-9e68-31f05f1516f3 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:41.591621 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfcbf35-49f1-40eb-baf4-dcaaa00f64c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:41.593477 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfcbf35-49f1-40eb-baf4-dcaaa00f64c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:41.593954 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfcbf35-49f1-40eb-baf4-dcaaa00f64c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:41.594336 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfcbf35-49f1-40eb-baf4-dcaaa00f64c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:41.595340 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fdf57-7e6a-4400-ba7e-220211213b11 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:41.596104 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fdf57-7e6a-4400-ba7e-220211213b11 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:41.596568 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fdf57-7e6a-4400-ba7e-220211213b11 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:41.601230 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 822/1644] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 00:08:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:08:41.611161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3e007-2df6-48ed-a76e-8c8ede715212 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:41.611646 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3e007-2df6-48ed-a76e-8c8ede715212 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:41.612168 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3e007-2df6-48ed-a76e-8c8ede715212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:41.618127 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfcbf35-49f1-40eb-baf4-dcaaa00f64c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:41.619460 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfcbf35-49f1-40eb-baf4-dcaaa00f64c5 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:41.654642 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 823/1645] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:41.656296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-f2e3e007-2df6-48ed-a76e-8c8ede715212 None None] MFA Rules not processed for user `c4a3579d56294b9d8ad36d4d174c8526`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:41.660765 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f2e3e007-2df6-48ed-a76e-8c8ede715212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:41.740150 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:41.771772 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 823/1646] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 00:08:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:08:41.793700 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfcbf35-49f1-40eb-baf4-dcaaa00f64c5 service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:41.795557 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5cfcbf35-49f1-40eb-baf4-dcaaa00f64c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:41.815957 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfcbf35-49f1-40eb-baf4-dcaaa00f64c5 service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:41.823040 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2d18d-3cef-48cb-b033-b9c52ccc9653 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:41.824248 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2d18d-3cef-48cb-b033-b9c52ccc9653 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:41.824896 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2d18d-3cef-48cb-b033-b9c52ccc9653 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:41.825502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a2d18d-3cef-48cb-b033-b9c52ccc9653 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:41.846575 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a2d18d-3cef-48cb-b033-b9c52ccc9653 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:41.848929 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a2d18d-3cef-48cb-b033-b9c52ccc9653 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:41.863594 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 824/1647] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:42.046490 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e3e007-2df6-48ed-a76e-8c8ede715212 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:42.048420 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f2e3e007-2df6-48ed-a76e-8c8ede715212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:42.111820 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e3e007-2df6-48ed-a76e-8c8ede715212 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:42.117585 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cacf555-55d2-4d08-903f-14f350e3e3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:42.118430 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cacf555-55d2-4d08-903f-14f350e3e3eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:42.118720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cacf555-55d2-4d08-903f-14f350e3e3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:42.119002 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cacf555-55d2-4d08-903f-14f350e3e3eb admin admin] PATH_INFO: `/v3/projects/dee07d424cad4b16b048043e765cd42c` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:42.120379 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cacf555-55d2-4d08-903f-14f350e3e3eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=dee07d424cad4b16b048043e765cd42c)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:42.122509 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cacf555-55d2-4d08-903f-14f350e3e3eb admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:42.169269 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5cacf555-55d2-4d08-903f-14f350e3e3eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dee07d424cad4b16b048043e765cd42c', 'request_id': 'req-5cacf555-55d2-4d08-903f-14f350e3e3eb'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:42.176873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5cacf555-55d2-4d08-903f-14f350e3e3eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dee07d424cad4b16b048043e765cd42c', 'request_id': 'req-5cacf555-55d2-4d08-903f-14f350e3e3eb'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:42.182432 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5cacf555-55d2-4d08-903f-14f350e3e3eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dee07d424cad4b16b048043e765cd42c', 'request_id': 'req-5cacf555-55d2-4d08-903f-14f350e3e3eb'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:42.194103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5cacf555-55d2-4d08-903f-14f350e3e3eb admin admin] The token cache is being invalidate because project dee07d424cad4b16b048043e765cd42c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:42.194420 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5cacf555-55d2-4d08-903f-14f350e3e3eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:42.195049 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5cacf555-55d2-4d08-903f-14f350e3e3eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:42.196532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 824/1648] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:42 2022] DELETE /identity/v3/projects/dee07d424cad4b16b048043e765cd42c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:42.200503 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2d18d-3cef-48cb-b033-b9c52ccc9653 service nova] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:42.211010 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a2d18d-3cef-48cb-b033-b9c52ccc9653 service nova] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:42.267043 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d80f69-8094-410f-b991-3e18a40f9509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:42.267667 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d80f69-8094-410f-b991-3e18a40f9509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:42.267950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d80f69-8094-410f-b991-3e18a40f9509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:42.268255 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d80f69-8094-410f-b991-3e18a40f9509 admin admin] PATH_INFO: `/v3/users/bc211296568947f69bab800cbbd1aaed` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:42.269398 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d80f69-8094-410f-b991-3e18a40f9509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc211296568947f69bab800cbbd1aaed)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:42.271307 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d80f69-8094-410f-b991-3e18a40f9509 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:42.317660 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-43d80f69-8094-410f-b991-3e18a40f9509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc211296568947f69bab800cbbd1aaed'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:42.324290 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-43d80f69-8094-410f-b991-3e18a40f9509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc211296568947f69bab800cbbd1aaed'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:42.335947 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-43d80f69-8094-410f-b991-3e18a40f9509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc211296568947f69bab800cbbd1aaed'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:42.336629 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-43d80f69-8094-410f-b991-3e18a40f9509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc211296568947f69bab800cbbd1aaed'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:42.337227 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-43d80f69-8094-410f-b991-3e18a40f9509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc211296568947f69bab800cbbd1aaed'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:42.340777 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:42.352842 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 825/1649] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:42 2022] DELETE /identity/v3/users/bc211296568947f69bab800cbbd1aaed => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:42.714530 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cacf555-55d2-4d08-903f-14f350e3e3eb admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:42.717117 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-5cacf555-55d2-4d08-903f-14f350e3e3eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:42.734279 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cacf555-55d2-4d08-903f-14f350e3e3eb admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:42.812587 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e1466-8ac0-4873-b167-8d9ec98e1873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:42.812889 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e1466-8ac0-4873-b167-8d9ec98e1873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:42.813181 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e1466-8ac0-4873-b167-8d9ec98e1873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:42.813463 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e1466-8ac0-4873-b167-8d9ec98e1873 admin admin] PATH_INFO: `/v3/projects/237d8f3f57be4b0fb27f8392808f867b` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:42.814785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e1466-8ac0-4873-b167-8d9ec98e1873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=237d8f3f57be4b0fb27f8392808f867b)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:42.817468 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e1466-8ac0-4873-b167-8d9ec98e1873 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:42.883298 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fe1e1466-8ac0-4873-b167-8d9ec98e1873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '237d8f3f57be4b0fb27f8392808f867b', 'request_id': 'req-fe1e1466-8ac0-4873-b167-8d9ec98e1873'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:42.893702 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fe1e1466-8ac0-4873-b167-8d9ec98e1873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '237d8f3f57be4b0fb27f8392808f867b', 'request_id': 'req-fe1e1466-8ac0-4873-b167-8d9ec98e1873'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:42.899850 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fe1e1466-8ac0-4873-b167-8d9ec98e1873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '237d8f3f57be4b0fb27f8392808f867b', 'request_id': 'req-fe1e1466-8ac0-4873-b167-8d9ec98e1873'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:42.911523 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fe1e1466-8ac0-4873-b167-8d9ec98e1873 admin admin] The token cache is being invalidate because project 237d8f3f57be4b0fb27f8392808f867b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:42.911808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fe1e1466-8ac0-4873-b167-8d9ec98e1873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:42.912498 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fe1e1466-8ac0-4873-b167-8d9ec98e1873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:42.917001 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 825/1650] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:42 2022] DELETE /identity/v3/projects/237d8f3f57be4b0fb27f8392808f867b => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:42.918235 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d80f69-8094-410f-b991-3e18a40f9509 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:42.919754 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-43d80f69-8094-410f-b991-3e18a40f9509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:42.935866 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d80f69-8094-410f-b991-3e18a40f9509 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:43.009411 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ae2b1-29cf-4672-94c0-4f42177afa6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:43.010052 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ae2b1-29cf-4672-94c0-4f42177afa6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:43.010335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ae2b1-29cf-4672-94c0-4f42177afa6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:43.010611 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ae2b1-29cf-4672-94c0-4f42177afa6f admin admin] PATH_INFO: `/v3/users/f363a0851c4f46fb9322a5f14108b8a5` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:43.011733 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ae2b1-29cf-4672-94c0-4f42177afa6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f363a0851c4f46fb9322a5f14108b8a5)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:43.013665 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ae2b1-29cf-4672-94c0-4f42177afa6f admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:43.061081 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-703ae2b1-29cf-4672-94c0-4f42177afa6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f363a0851c4f46fb9322a5f14108b8a5'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:43.068500 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-703ae2b1-29cf-4672-94c0-4f42177afa6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f363a0851c4f46fb9322a5f14108b8a5'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:43.080862 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-703ae2b1-29cf-4672-94c0-4f42177afa6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f363a0851c4f46fb9322a5f14108b8a5'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:43.081473 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-703ae2b1-29cf-4672-94c0-4f42177afa6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f363a0851c4f46fb9322a5f14108b8a5'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:43.082079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-703ae2b1-29cf-4672-94c0-4f42177afa6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f363a0851c4f46fb9322a5f14108b8a5'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:43.094455 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 826/1651] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:42 2022] DELETE /identity/v3/users/f363a0851c4f46fb9322a5f14108b8a5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:43.365461 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e1466-8ac0-4873-b167-8d9ec98e1873 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:43.366885 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-fe1e1466-8ac0-4873-b167-8d9ec98e1873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:43.376853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:43.388273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e1466-8ac0-4873-b167-8d9ec98e1873 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:43.444308 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e179f-43f0-4d28-a699-e395c8aeb7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:43.444974 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e179f-43f0-4d28-a699-e395c8aeb7aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:43.445261 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e179f-43f0-4d28-a699-e395c8aeb7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:43.445540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e179f-43f0-4d28-a699-e395c8aeb7aa admin admin] PATH_INFO: `/v3/projects/ee52f9e94e4b4c289a4a31c10d0905d4` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:43.446666 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e179f-43f0-4d28-a699-e395c8aeb7aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee52f9e94e4b4c289a4a31c10d0905d4)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:43.448594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e179f-43f0-4d28-a699-e395c8aeb7aa admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:43.499487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-112e179f-43f0-4d28-a699-e395c8aeb7aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee52f9e94e4b4c289a4a31c10d0905d4', 'request_id': 'req-112e179f-43f0-4d28-a699-e395c8aeb7aa'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:43.506690 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-112e179f-43f0-4d28-a699-e395c8aeb7aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee52f9e94e4b4c289a4a31c10d0905d4', 'request_id': 'req-112e179f-43f0-4d28-a699-e395c8aeb7aa'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:43.512849 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-112e179f-43f0-4d28-a699-e395c8aeb7aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee52f9e94e4b4c289a4a31c10d0905d4', 'request_id': 'req-112e179f-43f0-4d28-a699-e395c8aeb7aa'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:43.524499 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-112e179f-43f0-4d28-a699-e395c8aeb7aa admin admin] The token cache is being invalidate because project ee52f9e94e4b4c289a4a31c10d0905d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:43.524813 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-112e179f-43f0-4d28-a699-e395c8aeb7aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:43.525441 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-112e179f-43f0-4d28-a699-e395c8aeb7aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:43.526962 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 826/1652] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:43 2022] DELETE /identity/v3/projects/ee52f9e94e4b4c289a4a31c10d0905d4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:43.533923 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941d513-8cec-40cc-b17e-920e5742a624 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:43.534217 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941d513-8cec-40cc-b17e-920e5742a624 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:43.534506 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941d513-8cec-40cc-b17e-920e5742a624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:43.554208 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-b941d513-8cec-40cc-b17e-920e5742a624 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:43.555413 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b941d513-8cec-40cc-b17e-920e5742a624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:43.616175 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 827/1653] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:43.624890 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4640ea0-34e7-44f2-9f59-a94bb47bcf8c None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:43.625176 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4640ea0-34e7-44f2-9f59-a94bb47bcf8c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:43.625453 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4640ea0-34e7-44f2-9f59-a94bb47bcf8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:43.645543 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-b4640ea0-34e7-44f2-9f59-a94bb47bcf8c None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:43.646737 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b4640ea0-34e7-44f2-9f59-a94bb47bcf8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:43.654717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 827/1654] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:43.660133 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941d513-8cec-40cc-b17e-920e5742a624 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:43.670231 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941d513-8cec-40cc-b17e-920e5742a624 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:43.676037 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec61a6-03ca-4278-8874-e20f76dcb751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:43.676718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec61a6-03ca-4278-8874-e20f76dcb751 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:43.677030 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec61a6-03ca-4278-8874-e20f76dcb751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:43.677309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec61a6-03ca-4278-8874-e20f76dcb751 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:43.677959 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec61a6-03ca-4278-8874-e20f76dcb751 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:08:43.678264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec61a6-03ca-4278-8874-e20f76dcb751 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:43.679846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ec61a6-03ca-4278-8874-e20f76dcb751 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:43.695050 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 828/1655] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:08:43.696164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4640ea0-34e7-44f2-9f59-a94bb47bcf8c None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:43.706219 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4640ea0-34e7-44f2-9f59-a94bb47bcf8c None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:43.710904 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791eebfa-4f91-49ad-8f1b-4374de1b2475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:43.711519 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791eebfa-4f91-49ad-8f1b-4374de1b2475 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:43.711803 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791eebfa-4f91-49ad-8f1b-4374de1b2475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:43.712082 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791eebfa-4f91-49ad-8f1b-4374de1b2475 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:43.712759 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791eebfa-4f91-49ad-8f1b-4374de1b2475 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:43.714688 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791eebfa-4f91-49ad-8f1b-4374de1b2475 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:43.742823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 828/1656] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:43 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:43.747140 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec61a6-03ca-4278-8874-e20f76dcb751 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:43.753365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:43.763352 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec61a6-03ca-4278-8874-e20f76dcb751 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:43.822588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507445f-5ce4-4d81-a1ed-087aa254e4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:43.823230 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507445f-5ce4-4d81-a1ed-087aa254e4c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:43.823510 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507445f-5ce4-4d81-a1ed-087aa254e4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:43.823810 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507445f-5ce4-4d81-a1ed-087aa254e4c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:43.824443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507445f-5ce4-4d81-a1ed-087aa254e4c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:43.826425 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507445f-5ce4-4d81-a1ed-087aa254e4c5 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:43.851935 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 829/1657] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:43 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:08:43.853553 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791eebfa-4f91-49ad-8f1b-4374de1b2475 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:43.866910 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791eebfa-4f91-49ad-8f1b-4374de1b2475 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:43.871810 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb545e-af47-4eda-aeb2-3d22e510358f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:43.872443 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb545e-af47-4eda-aeb2-3d22e510358f admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:43.872777 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb545e-af47-4eda-aeb2-3d22e510358f admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:43.873075 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb545e-af47-4eda-aeb2-3d22e510358f admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:43.873818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb545e-af47-4eda-aeb2-3d22e510358f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:43.875560 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb545e-af47-4eda-aeb2-3d22e510358f admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:43.888556 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 829/1658] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:43.890297 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507445f-5ce4-4d81-a1ed-087aa254e4c5 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:43.901225 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507445f-5ce4-4d81-a1ed-087aa254e4c5 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:43.906742 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f3aeb0-3257-41af-81a9-fa1e4420917c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:43.907424 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3aeb0-3257-41af-81a9-fa1e4420917c admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:43.907777 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3aeb0-3257-41af-81a9-fa1e4420917c admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:43.908132 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3aeb0-3257-41af-81a9-fa1e4420917c admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:43.908863 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f3aeb0-3257-41af-81a9-fa1e4420917c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:43.910521 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f3aeb0-3257-41af-81a9-fa1e4420917c admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:43.921752 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 830/1659] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:43.926261 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb545e-af47-4eda-aeb2-3d22e510358f admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:43.939514 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb545e-af47-4eda-aeb2-3d22e510358f admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:43.945937 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa608c-7123-43a8-95b9-5715c71e3c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:43.947739 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa608c-7123-43a8-95b9-5715c71e3c42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:43.947739 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa608c-7123-43a8-95b9-5715c71e3c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:43.947739 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa608c-7123-43a8-95b9-5715c71e3c42 admin admin] PATH_INFO: `/v3/projects/268a58134e11444a8f34c7f1ffd304d6/users/88501c890a304ad6a96e4d6203fb435e/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:43.977281 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa608c-7123-43a8-95b9-5715c71e3c42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=268a58134e11444a8f34c7f1ffd304d6, user_id=88501c890a304ad6a96e4d6203fb435e, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:43.980548 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa608c-7123-43a8-95b9-5715c71e3c42 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:44.011874 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 830/1660] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:43 2022] PUT /identity/v3/projects/268a58134e11444a8f34c7f1ffd304d6/users/88501c890a304ad6a96e4d6203fb435e/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:44.013636 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427a21e-c966-43c5-8164-8e1208a156b0 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:44.014099 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427a21e-c966-43c5-8164-8e1208a156b0 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:44.014696 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427a21e-c966-43c5-8164-8e1208a156b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:44.060740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-3427a21e-c966-43c5-8164-8e1208a156b0 None None] MFA Rules not processed for user `88501c890a304ad6a96e4d6203fb435e`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:44.060740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-3427a21e-c966-43c5-8164-8e1208a156b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:44.131524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 831/1661] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:08:44.136052 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa608c-7123-43a8-95b9-5715c71e3c42 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:44.171748 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:44.212765 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa608c-7123-43a8-95b9-5715c71e3c42 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:44.218361 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba78fb2-7246-4809-83eb-220539614f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:44.219051 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba78fb2-7246-4809-83eb-220539614f87 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:44.219518 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba78fb2-7246-4809-83eb-220539614f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:44.219890 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba78fb2-7246-4809-83eb-220539614f87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:44.220520 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba78fb2-7246-4809-83eb-220539614f87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:44.222429 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba78fb2-7246-4809-83eb-220539614f87 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:44.256273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 831/1662] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:44 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:44.260520 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427a21e-c966-43c5-8164-8e1208a156b0 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:44.271831 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427a21e-c966-43c5-8164-8e1208a156b0 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:44.328391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdf4bd-784c-48c5-8ba4-802dcdcf7f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:44.329075 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdf4bd-784c-48c5-8ba4-802dcdcf7f7a admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:44.329365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdf4bd-784c-48c5-8ba4-802dcdcf7f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:44.329647 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfdf4bd-784c-48c5-8ba4-802dcdcf7f7a admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:44.330324 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfdf4bd-784c-48c5-8ba4-802dcdcf7f7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:44.332235 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfdf4bd-784c-48c5-8ba4-802dcdcf7f7a admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:44.350814 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:44.360524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 832/1663] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:44 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:08:44.361457 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba78fb2-7246-4809-83eb-220539614f87 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:44.372538 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba78fb2-7246-4809-83eb-220539614f87 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:44.377563 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d7561-04d8-49da-bf86-5a5486c7a7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:44.378172 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d7561-04d8-49da-bf86-5a5486c7a7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:44.378453 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d7561-04d8-49da-bf86-5a5486c7a7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:44.378738 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d7561-04d8-49da-bf86-5a5486c7a7a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:44.379383 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d7561-04d8-49da-bf86-5a5486c7a7a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:44.381165 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d7561-04d8-49da-bf86-5a5486c7a7a9 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:44.394201 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 832/1664] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:08:44.398598 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdf4bd-784c-48c5-8ba4-802dcdcf7f7a admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:44.409118 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfdf4bd-784c-48c5-8ba4-802dcdcf7f7a admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:44.414119 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33045ee8-bf4f-4fd6-a138-83edde19b5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:44.414899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33045ee8-bf4f-4fd6-a138-83edde19b5c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:44.415181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33045ee8-bf4f-4fd6-a138-83edde19b5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:44.415460 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33045ee8-bf4f-4fd6-a138-83edde19b5c6 admin admin] PATH_INFO: `/v3/projects/7d9c4e0a9bb247feb49f9c7ef6393ca4/users/6b689f64354f4ed99d56172a05b23725/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:44.430933 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33045ee8-bf4f-4fd6-a138-83edde19b5c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d9c4e0a9bb247feb49f9c7ef6393ca4, user_id=6b689f64354f4ed99d56172a05b23725, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:44.433629 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33045ee8-bf4f-4fd6-a138-83edde19b5c6 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:44.446994 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 833/1665] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:44 2022] PUT /identity/v3/projects/7d9c4e0a9bb247feb49f9c7ef6393ca4/users/6b689f64354f4ed99d56172a05b23725/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:08:44.451498 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d7561-04d8-49da-bf86-5a5486c7a7a9 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:44.464134 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d7561-04d8-49da-bf86-5a5486c7a7a9 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:44.548302 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9247a-1d34-4fd5-b3bf-b03acc7e36f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:44.548542 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9247a-1d34-4fd5-b3bf-b03acc7e36f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:44.548673 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9247a-1d34-4fd5-b3bf-b03acc7e36f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:44.548952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9247a-1d34-4fd5-b3bf-b03acc7e36f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:44.549592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f9247a-1d34-4fd5-b3bf-b03acc7e36f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:44.551219 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f9247a-1d34-4fd5-b3bf-b03acc7e36f1 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:44.564039 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 833/1666] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:44.567676 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33045ee8-bf4f-4fd6-a138-83edde19b5c6 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:44.580224 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33045ee8-bf4f-4fd6-a138-83edde19b5c6 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:44.585499 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b69fc6-d9e8-4b3c-bd0a-8938418f2e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:44.587063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b69fc6-d9e8-4b3c-bd0a-8938418f2e64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:44.587425 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b69fc6-d9e8-4b3c-bd0a-8938418f2e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:44.587779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b69fc6-d9e8-4b3c-bd0a-8938418f2e64 admin admin] PATH_INFO: `/v3/projects/7d9c4e0a9bb247feb49f9c7ef6393ca4/users/6b689f64354f4ed99d56172a05b23725/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:44.594434 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b69fc6-d9e8-4b3c-bd0a-8938418f2e64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d9c4e0a9bb247feb49f9c7ef6393ca4, user_id=6b689f64354f4ed99d56172a05b23725, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:44.597078 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b69fc6-d9e8-4b3c-bd0a-8938418f2e64 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:44.609449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 834/1667] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:44 2022] PUT /identity/v3/projects/7d9c4e0a9bb247feb49f9c7ef6393ca4/users/6b689f64354f4ed99d56172a05b23725/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:08:44.619598 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c061004-b232-4526-a059-40cb4dbf819f None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:44.619891 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c061004-b232-4526-a059-40cb4dbf819f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:44.620172 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c061004-b232-4526-a059-40cb4dbf819f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:44.648448 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-4c061004-b232-4526-a059-40cb4dbf819f None None] MFA Rules not processed for user `6b689f64354f4ed99d56172a05b23725`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:44.649610 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-4c061004-b232-4526-a059-40cb4dbf819f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:44.731154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 834/1668] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:44 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:44.742200 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b69fc6-d9e8-4b3c-bd0a-8938418f2e64 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:44.743818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-c4b69fc6-d9e8-4b3c-bd0a-8938418f2e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:44.805002 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b69fc6-d9e8-4b3c-bd0a-8938418f2e64 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:44.808658 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849cba01-2f32-4b32-afbb-de0bb0b7d3d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:44.809264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849cba01-2f32-4b32-afbb-de0bb0b7d3d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:44.809546 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849cba01-2f32-4b32-afbb-de0bb0b7d3d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:44.809822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849cba01-2f32-4b32-afbb-de0bb0b7d3d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:44.869326 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849cba01-2f32-4b32-afbb-de0bb0b7d3d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:44.870796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849cba01-2f32-4b32-afbb-de0bb0b7d3d5 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:44.880814 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 835/1669] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:49.776705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c061004-b232-4526-a059-40cb4dbf819f None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:49.788825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-4c061004-b232-4526-a059-40cb4dbf819f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:49.847580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:49.892403 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c061004-b232-4526-a059-40cb4dbf819f None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:49.897547 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b4cbb2-e2ab-465c-9169-5e738dee118b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:49.898192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b4cbb2-e2ab-465c-9169-5e738dee118b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:49.898615 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b4cbb2-e2ab-465c-9169-5e738dee118b admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:49.898759 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b4cbb2-e2ab-465c-9169-5e738dee118b admin admin] PATH_INFO: `/v3/users/c3823a85a5d54c9e8a4a2556e73a2c81` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:49.899890 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b4cbb2-e2ab-465c-9169-5e738dee118b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3823a85a5d54c9e8a4a2556e73a2c81)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:49.904398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b4cbb2-e2ab-465c-9169-5e738dee118b admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:49.968733 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-97b4cbb2-e2ab-465c-9169-5e738dee118b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3823a85a5d54c9e8a4a2556e73a2c81'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:49.969471 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-97b4cbb2-e2ab-465c-9169-5e738dee118b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3823a85a5d54c9e8a4a2556e73a2c81'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:49.976848 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-97b4cbb2-e2ab-465c-9169-5e738dee118b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3823a85a5d54c9e8a4a2556e73a2c81'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:49.989709 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-97b4cbb2-e2ab-465c-9169-5e738dee118b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3823a85a5d54c9e8a4a2556e73a2c81'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:49.997317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-97b4cbb2-e2ab-465c-9169-5e738dee118b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3823a85a5d54c9e8a4a2556e73a2c81'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:50.004576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 835/1670] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:49 2022] DELETE /identity/v3/users/c3823a85a5d54c9e8a4a2556e73a2c81 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:50.009275 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d681ea-0947-448e-ad84-a1a7dfa3953e None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:50.009747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d681ea-0947-448e-ad84-a1a7dfa3953e None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:50.009857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d681ea-0947-448e-ad84-a1a7dfa3953e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:50.030794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-e8d681ea-0947-448e-ad84-a1a7dfa3953e None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:50.031710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e8d681ea-0947-448e-ad84-a1a7dfa3953e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:50.114999 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 836/1671] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:08:50.125353 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b4cbb2-e2ab-465c-9169-5e738dee118b admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:50.127003 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-97b4cbb2-e2ab-465c-9169-5e738dee118b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:50.197553 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b4cbb2-e2ab-465c-9169-5e738dee118b admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:50.210347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89346de-c945-4568-a9fc-e6de4ea9ff3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:50.211150 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89346de-c945-4568-a9fc-e6de4ea9ff3a service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:50.211439 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89346de-c945-4568-a9fc-e6de4ea9ff3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:50.211745 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89346de-c945-4568-a9fc-e6de4ea9ff3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:50.221916 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89346de-c945-4568-a9fc-e6de4ea9ff3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:50.223352 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89346de-c945-4568-a9fc-e6de4ea9ff3a service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:50.248563 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 836/1672] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:50.584941 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d681ea-0947-448e-ad84-a1a7dfa3953e None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:50.586675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e8d681ea-0947-448e-ad84-a1a7dfa3953e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:50.601746 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d681ea-0947-448e-ad84-a1a7dfa3953e None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:50.606828 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbce822-8d2d-41e0-80c1-8a4c468605fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:50.607472 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbce822-8d2d-41e0-80c1-8a4c468605fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:50.607756 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbce822-8d2d-41e0-80c1-8a4c468605fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:50.608030 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbce822-8d2d-41e0-80c1-8a4c468605fa admin admin] PATH_INFO: `/v3/projects/eff745c2bd5f4dfa825ef57cf81c3ae2` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:50.609923 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbce822-8d2d-41e0-80c1-8a4c468605fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=eff745c2bd5f4dfa825ef57cf81c3ae2)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:50.612000 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbce822-8d2d-41e0-80c1-8a4c468605fa admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:50.625244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:50.662859 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-2dbce822-8d2d-41e0-80c1-8a4c468605fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eff745c2bd5f4dfa825ef57cf81c3ae2', 'request_id': 'req-2dbce822-8d2d-41e0-80c1-8a4c468605fa'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:50.669144 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-2dbce822-8d2d-41e0-80c1-8a4c468605fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eff745c2bd5f4dfa825ef57cf81c3ae2', 'request_id': 'req-2dbce822-8d2d-41e0-80c1-8a4c468605fa'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:50.677646 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-2dbce822-8d2d-41e0-80c1-8a4c468605fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eff745c2bd5f4dfa825ef57cf81c3ae2', 'request_id': 'req-2dbce822-8d2d-41e0-80c1-8a4c468605fa'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:50.689481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-2dbce822-8d2d-41e0-80c1-8a4c468605fa admin admin] The token cache is being invalidate because project eff745c2bd5f4dfa825ef57cf81c3ae2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:50.689808 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-2dbce822-8d2d-41e0-80c1-8a4c468605fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:50.690453 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-2dbce822-8d2d-41e0-80c1-8a4c468605fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:50.691958 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 837/1673] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:50 2022] DELETE /identity/v3/projects/eff745c2bd5f4dfa825ef57cf81c3ae2 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:50.695888 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89346de-c945-4568-a9fc-e6de4ea9ff3a service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:50.706475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89346de-c945-4568-a9fc-e6de4ea9ff3a service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:50.755497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:50.771611 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f7850-3eb2-4f3a-bdbd-2f26eede5452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:50.772337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f7850-3eb2-4f3a-bdbd-2f26eede5452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:50.772725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f7850-3eb2-4f3a-bdbd-2f26eede5452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:50.773091 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f7850-3eb2-4f3a-bdbd-2f26eede5452 admin admin] PATH_INFO: `/v3/users/b2e0ea51339048e6bcc11c02f6257b03` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:50.774309 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854f7850-3eb2-4f3a-bdbd-2f26eede5452 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2e0ea51339048e6bcc11c02f6257b03)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:50.776361 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854f7850-3eb2-4f3a-bdbd-2f26eede5452 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:50.821313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-854f7850-3eb2-4f3a-bdbd-2f26eede5452 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2e0ea51339048e6bcc11c02f6257b03'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:50.821986 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-854f7850-3eb2-4f3a-bdbd-2f26eede5452 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2e0ea51339048e6bcc11c02f6257b03'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:50.829598 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-854f7850-3eb2-4f3a-bdbd-2f26eede5452 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2e0ea51339048e6bcc11c02f6257b03'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:50.841655 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-854f7850-3eb2-4f3a-bdbd-2f26eede5452 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2e0ea51339048e6bcc11c02f6257b03'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:50.849914 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-854f7850-3eb2-4f3a-bdbd-2f26eede5452 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2e0ea51339048e6bcc11c02f6257b03'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:50.856400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 837/1674] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:50 2022] DELETE /identity/v3/users/b2e0ea51339048e6bcc11c02f6257b03 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:51.145152 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbce822-8d2d-41e0-80c1-8a4c468605fa admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:51.146590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-2dbce822-8d2d-41e0-80c1-8a4c468605fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:51.166365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbce822-8d2d-41e0-80c1-8a4c468605fa admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:51.227512 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f3eeb-eb82-4bbd-8563-511b066393b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:51.228158 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f3eeb-eb82-4bbd-8563-511b066393b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:51.228449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f3eeb-eb82-4bbd-8563-511b066393b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:51.228757 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f3eeb-eb82-4bbd-8563-511b066393b0 admin admin] PATH_INFO: `/v3/projects/9982af673646460b99e353d2392ec440` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:51.229903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f3eeb-eb82-4bbd-8563-511b066393b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9982af673646460b99e353d2392ec440)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:51.231790 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f3eeb-eb82-4bbd-8563-511b066393b0 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:51.305920 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-a38f3eeb-eb82-4bbd-8563-511b066393b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9982af673646460b99e353d2392ec440', 'request_id': 'req-a38f3eeb-eb82-4bbd-8563-511b066393b0'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:51.314479 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-a38f3eeb-eb82-4bbd-8563-511b066393b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9982af673646460b99e353d2392ec440', 'request_id': 'req-a38f3eeb-eb82-4bbd-8563-511b066393b0'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:51.322722 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-a38f3eeb-eb82-4bbd-8563-511b066393b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9982af673646460b99e353d2392ec440', 'request_id': 'req-a38f3eeb-eb82-4bbd-8563-511b066393b0'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:51.333360 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-a38f3eeb-eb82-4bbd-8563-511b066393b0 admin admin] The token cache is being invalidate because project 9982af673646460b99e353d2392ec440 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:51.333651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-a38f3eeb-eb82-4bbd-8563-511b066393b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:51.334300 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-a38f3eeb-eb82-4bbd-8563-511b066393b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:51.335801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 838/1675] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:51 2022] DELETE /identity/v3/projects/9982af673646460b99e353d2392ec440 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:51.340877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ef17c-ddeb-41ae-9b87-82c5411331a0 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:51.341169 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ef17c-ddeb-41ae-9b87-82c5411331a0 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:51.341451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ef17c-ddeb-41ae-9b87-82c5411331a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:51.361739 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-f57ef17c-ddeb-41ae-9b87-82c5411331a0 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:51.362964 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f57ef17c-ddeb-41ae-9b87-82c5411331a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:51.433440 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 838/1676] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:08:51.442938 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed9229-2c57-4c9e-94d7-1efd3a2ca1a1 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:51.443231 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed9229-2c57-4c9e-94d7-1efd3a2ca1a1 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:51.443539 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed9229-2c57-4c9e-94d7-1efd3a2ca1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:51.465526 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-51ed9229-2c57-4c9e-94d7-1efd3a2ca1a1 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:51.466731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-51ed9229-2c57-4c9e-94d7-1efd3a2ca1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:51.475590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 839/1677] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:51.479916 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ef17c-ddeb-41ae-9b87-82c5411331a0 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:51.489506 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ef17c-ddeb-41ae-9b87-82c5411331a0 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:51.494054 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e1e10-d739-4881-88f3-4cff02dac435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:51.494649 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e1e10-d739-4881-88f3-4cff02dac435 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:51.494933 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e1e10-d739-4881-88f3-4cff02dac435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:51.495210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e1e10-d739-4881-88f3-4cff02dac435 admin admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:51.495851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e1e10-d739-4881-88f3-4cff02dac435 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:08:51.496159 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e1e10-d739-4881-88f3-4cff02dac435 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:51.497787 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e1e10-d739-4881-88f3-4cff02dac435 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:51.510098 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 839/1678] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:08:51.514163 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed9229-2c57-4c9e-94d7-1efd3a2ca1a1 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:51.523993 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed9229-2c57-4c9e-94d7-1efd3a2ca1a1 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:51.528627 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0e02d-50b8-4e31-9fa6-6ab771f0b0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:51.529225 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0e02d-50b8-4e31-9fa6-6ab771f0b0ff admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:51.529528 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0e02d-50b8-4e31-9fa6-6ab771f0b0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:51.529790 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa0e02d-50b8-4e31-9fa6-6ab771f0b0ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:51.530480 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa0e02d-50b8-4e31-9fa6-6ab771f0b0ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:51.532677 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa0e02d-50b8-4e31-9fa6-6ab771f0b0ff admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:51.559221 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 840/1679] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:51 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:51.564405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e1e10-d739-4881-88f3-4cff02dac435 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:51.574807 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e1e10-d739-4881-88f3-4cff02dac435 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:51.633068 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:51.649268 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f4e6e8-6bea-430b-be16-fd90c2a0d29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:51.650007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4e6e8-6bea-430b-be16-fd90c2a0d29e admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:51.650348 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4e6e8-6bea-430b-be16-fd90c2a0d29e admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:51.650718 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4e6e8-6bea-430b-be16-fd90c2a0d29e admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:51.651931 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f4e6e8-6bea-430b-be16-fd90c2a0d29e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:51.654194 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f4e6e8-6bea-430b-be16-fd90c2a0d29e admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:51.681922 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 840/1680] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:51 2022] POST /identity/v3/users => generated 285 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:51.683219 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0e02d-50b8-4e31-9fa6-6ab771f0b0ff admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:51.690490 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:51.697485 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa0e02d-50b8-4e31-9fa6-6ab771f0b0ff admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:51.704749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa4f9e-83b6-4253-b799-048d531f0b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:51.705368 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa4f9e-83b6-4253-b799-048d531f0b08 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:51.705652 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa4f9e-83b6-4253-b799-048d531f0b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:51.705927 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa4f9e-83b6-4253-b799-048d531f0b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:51.706571 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa4f9e-83b6-4253-b799-048d531f0b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:51.708202 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa4f9e-83b6-4253-b799-048d531f0b08 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:51.719397 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 841/1681] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:51.723304 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f4e6e8-6bea-430b-be16-fd90c2a0d29e admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:51.733946 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f4e6e8-6bea-430b-be16-fd90c2a0d29e admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:51.739070 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c54edfa-76c0-455e-bb6f-d78f365e9daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:51.739743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54edfa-76c0-455e-bb6f-d78f365e9daf admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:51.740029 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54edfa-76c0-455e-bb6f-d78f365e9daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:51.740328 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c54edfa-76c0-455e-bb6f-d78f365e9daf admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:51.741012 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54edfa-76c0-455e-bb6f-d78f365e9daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:51.742631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c54edfa-76c0-455e-bb6f-d78f365e9daf admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:51.756691 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 841/1682] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:51.760936 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa4f9e-83b6-4253-b799-048d531f0b08 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:51.777237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa4f9e-83b6-4253-b799-048d531f0b08 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:51.785888 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd7793-6168-43ab-aba3-21540e5df6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:51.786544 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd7793-6168-43ab-aba3-21540e5df6d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:51.786822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd7793-6168-43ab-aba3-21540e5df6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:51.787097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd7793-6168-43ab-aba3-21540e5df6d0 admin admin] PATH_INFO: `/v3/projects/d582c390822740d181e71c12c8bc1562/users/91fc6a767e31408c8793fb9cd7b5134f/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:51.803050 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd7793-6168-43ab-aba3-21540e5df6d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d582c390822740d181e71c12c8bc1562, user_id=91fc6a767e31408c8793fb9cd7b5134f, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:51.805689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd7793-6168-43ab-aba3-21540e5df6d0 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:51.820293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 842/1683] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:51 2022] PUT /identity/v3/projects/d582c390822740d181e71c12c8bc1562/users/91fc6a767e31408c8793fb9cd7b5134f/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:08:51.831913 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15d9eb-f69d-4b4b-bafc-ffb85ceef25d None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:51.832416 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15d9eb-f69d-4b4b-bafc-ffb85ceef25d None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:51.832742 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15d9eb-f69d-4b4b-bafc-ffb85ceef25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:51.861511 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-9a15d9eb-f69d-4b4b-bafc-ffb85ceef25d None None] MFA Rules not processed for user `91fc6a767e31408c8793fb9cd7b5134f`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:51.862726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9a15d9eb-f69d-4b4b-bafc-ffb85ceef25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:51.930184 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 842/1684] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:51.941759 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd7793-6168-43ab-aba3-21540e5df6d0 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:52.004534 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd7793-6168-43ab-aba3-21540e5df6d0 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:52.009246 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc4d7bc-4b0d-4bd5-b639-532694681961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:52.009849 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4d7bc-4b0d-4bd5-b639-532694681961 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:52.010133 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4d7bc-4b0d-4bd5-b639-532694681961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:52.010409 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4d7bc-4b0d-4bd5-b639-532694681961 admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:52.011049 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc4d7bc-4b0d-4bd5-b639-532694681961 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:52.012982 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc4d7bc-4b0d-4bd5-b639-532694681961 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:52.040699 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 843/1685] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:51 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:52.044737 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15d9eb-f69d-4b4b-bafc-ffb85ceef25d None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:52.055809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15d9eb-f69d-4b4b-bafc-ffb85ceef25d None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:52.127427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760ff1f-2271-4e0d-bb31-13374b9fc1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:52.128051 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760ff1f-2271-4e0d-bb31-13374b9fc1f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:52.128336 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760ff1f-2271-4e0d-bb31-13374b9fc1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:52.128696 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760ff1f-2271-4e0d-bb31-13374b9fc1f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:52.129340 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760ff1f-2271-4e0d-bb31-13374b9fc1f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:52.131245 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760ff1f-2271-4e0d-bb31-13374b9fc1f6 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:52.154572 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 843/1686] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:52 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:52.158468 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc4d7bc-4b0d-4bd5-b639-532694681961 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:52.176109 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc4d7bc-4b0d-4bd5-b639-532694681961 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:52.181006 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ea038-a7b0-4d3f-ace6-c856a8bf32ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:52.181660 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ea038-a7b0-4d3f-ace6-c856a8bf32ef admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:52.181948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ea038-a7b0-4d3f-ace6-c856a8bf32ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:52.182227 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ea038-a7b0-4d3f-ace6-c856a8bf32ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:52.182868 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ea038-a7b0-4d3f-ace6-c856a8bf32ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:52.184458 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ea038-a7b0-4d3f-ace6-c856a8bf32ef admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:52.195945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 844/1687] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:52.200210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760ff1f-2271-4e0d-bb31-13374b9fc1f6 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:52.210257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760ff1f-2271-4e0d-bb31-13374b9fc1f6 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:52.214826 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d6ee4-4a3f-444e-9c9a-c28cee719dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:52.215478 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d6ee4-4a3f-444e-9c9a-c28cee719dd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:52.215763 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d6ee4-4a3f-444e-9c9a-c28cee719dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:52.216045 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990d6ee4-4a3f-444e-9c9a-c28cee719dd5 admin admin] PATH_INFO: `/v3/projects/3ad7c30d41e241058ee1e9c3fe53b81e/users/d1aee11e71454d74bec36a075c777dfa/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:52.232274 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d6ee4-4a3f-444e-9c9a-c28cee719dd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ad7c30d41e241058ee1e9c3fe53b81e, user_id=d1aee11e71454d74bec36a075c777dfa, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:52.235699 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990d6ee4-4a3f-444e-9c9a-c28cee719dd5 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:52.248556 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 844/1688] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:52 2022] PUT /identity/v3/projects/3ad7c30d41e241058ee1e9c3fe53b81e/users/d1aee11e71454d74bec36a075c777dfa/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:52.252452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ea038-a7b0-4d3f-ace6-c856a8bf32ef admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:52.262246 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ea038-a7b0-4d3f-ace6-c856a8bf32ef admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:52.318165 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720197bc-9827-42e3-816c-c3709e79f148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:52.318775 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720197bc-9827-42e3-816c-c3709e79f148 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:52.319054 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720197bc-9827-42e3-816c-c3709e79f148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:52.319330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720197bc-9827-42e3-816c-c3709e79f148 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:52.319965 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720197bc-9827-42e3-816c-c3709e79f148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:52.321565 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720197bc-9827-42e3-816c-c3709e79f148 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:52.335285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 845/1689] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:52.336972 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d6ee4-4a3f-444e-9c9a-c28cee719dd5 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:52.348357 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990d6ee4-4a3f-444e-9c9a-c28cee719dd5 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:52.352995 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef051e2b-5744-4b27-9182-1013e701aacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:52.353653 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef051e2b-5744-4b27-9182-1013e701aacf admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:52.353937 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef051e2b-5744-4b27-9182-1013e701aacf admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:52.354212 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef051e2b-5744-4b27-9182-1013e701aacf admin admin] PATH_INFO: `/v3/projects/3ad7c30d41e241058ee1e9c3fe53b81e/users/d1aee11e71454d74bec36a075c777dfa/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:52.356154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef051e2b-5744-4b27-9182-1013e701aacf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ad7c30d41e241058ee1e9c3fe53b81e, user_id=d1aee11e71454d74bec36a075c777dfa, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:52.358861 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef051e2b-5744-4b27-9182-1013e701aacf admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:52.369095 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 845/1690] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:52 2022] PUT /identity/v3/projects/3ad7c30d41e241058ee1e9c3fe53b81e/users/d1aee11e71454d74bec36a075c777dfa/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:52.379661 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384bd4d-bfd6-44c3-b738-70d2109177c5 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:52.379961 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384bd4d-bfd6-44c3-b738-70d2109177c5 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:52.380240 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384bd4d-bfd6-44c3-b738-70d2109177c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:52.408805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-a384bd4d-bfd6-44c3-b738-70d2109177c5 None None] MFA Rules not processed for user `d1aee11e71454d74bec36a075c777dfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:52.409999 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a384bd4d-bfd6-44c3-b738-70d2109177c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:52.436557 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:52.480168 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 846/1691] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:52 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:52.491392 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef051e2b-5744-4b27-9182-1013e701aacf admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:52.493146 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ef051e2b-5744-4b27-9182-1013e701aacf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:52.512340 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:52.561982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef051e2b-5744-4b27-9182-1013e701aacf admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:52.565561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df28417-5f1a-4393-8301-87ca1e195306 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:52.566160 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df28417-5f1a-4393-8301-87ca1e195306 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:52.566444 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df28417-5f1a-4393-8301-87ca1e195306 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:52.566722 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df28417-5f1a-4393-8301-87ca1e195306 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:52.575464 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df28417-5f1a-4393-8301-87ca1e195306 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:52.576916 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df28417-5f1a-4393-8301-87ca1e195306 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:52.586635 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 846/1692] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:52 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:53.983181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384bd4d-bfd6-44c3-b738-70d2109177c5 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:54.049649 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384bd4d-bfd6-44c3-b738-70d2109177c5 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:54.054194 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a04a91-fcab-4549-a26b-cc100612bdcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:54.054830 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a04a91-fcab-4549-a26b-cc100612bdcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:54.055111 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a04a91-fcab-4549-a26b-cc100612bdcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:54.055384 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a04a91-fcab-4549-a26b-cc100612bdcc admin admin] PATH_INFO: `/v3/users/91fc6a767e31408c8793fb9cd7b5134f` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:54.056652 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a04a91-fcab-4549-a26b-cc100612bdcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=91fc6a767e31408c8793fb9cd7b5134f)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:54.058814 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a04a91-fcab-4549-a26b-cc100612bdcc admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:54.111037 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-79a04a91-fcab-4549-a26b-cc100612bdcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91fc6a767e31408c8793fb9cd7b5134f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:54.120042 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-79a04a91-fcab-4549-a26b-cc100612bdcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91fc6a767e31408c8793fb9cd7b5134f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:54.135794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-79a04a91-fcab-4549-a26b-cc100612bdcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91fc6a767e31408c8793fb9cd7b5134f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:54.136558 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-79a04a91-fcab-4549-a26b-cc100612bdcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91fc6a767e31408c8793fb9cd7b5134f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:54.137784 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-79a04a91-fcab-4549-a26b-cc100612bdcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91fc6a767e31408c8793fb9cd7b5134f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:54.156331 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 847/1693] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:53 2022] DELETE /identity/v3/users/91fc6a767e31408c8793fb9cd7b5134f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:54.168731 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc5f2d-0128-4c02-968f-d7e47317fc9d None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:54.168731 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc5f2d-0128-4c02-968f-d7e47317fc9d None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:54.168731 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fc5f2d-0128-4c02-968f-d7e47317fc9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:54.181847 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-88fc5f2d-0128-4c02-968f-d7e47317fc9d None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:54.183053 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-88fc5f2d-0128-4c02-968f-d7e47317fc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:54.253523 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 847/1694] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:54.263145 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a04a91-fcab-4549-a26b-cc100612bdcc admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:54.264872 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-79a04a91-fcab-4549-a26b-cc100612bdcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:54.315263 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:54.328616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a04a91-fcab-4549-a26b-cc100612bdcc admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:54.334124 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11565e99-de6b-4c42-8b74-6e75dfe0870f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:54.335273 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11565e99-de6b-4c42-8b74-6e75dfe0870f service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:54.335778 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11565e99-de6b-4c42-8b74-6e75dfe0870f service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:54.336309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11565e99-de6b-4c42-8b74-6e75dfe0870f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:54.347870 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11565e99-de6b-4c42-8b74-6e75dfe0870f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:54.349378 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11565e99-de6b-4c42-8b74-6e75dfe0870f service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:54.359809 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 848/1695] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:54.665662 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc5f2d-0128-4c02-968f-d7e47317fc9d None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:54.667326 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-88fc5f2d-0128-4c02-968f-d7e47317fc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:54.681233 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fc5f2d-0128-4c02-968f-d7e47317fc9d None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:54.687994 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57cb279-34b7-40c0-af25-396b62390d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:54.688813 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cb279-34b7-40c0-af25-396b62390d81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:54.689200 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cb279-34b7-40c0-af25-396b62390d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:54.689572 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cb279-34b7-40c0-af25-396b62390d81 admin admin] PATH_INFO: `/v3/projects/d582c390822740d181e71c12c8bc1562` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:54.692936 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57cb279-34b7-40c0-af25-396b62390d81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d582c390822740d181e71c12c8bc1562)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:54.695223 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57cb279-34b7-40c0-af25-396b62390d81 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:54.745669 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-e57cb279-34b7-40c0-af25-396b62390d81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd582c390822740d181e71c12c8bc1562', 'request_id': 'req-e57cb279-34b7-40c0-af25-396b62390d81'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:54.753966 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-e57cb279-34b7-40c0-af25-396b62390d81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd582c390822740d181e71c12c8bc1562', 'request_id': 'req-e57cb279-34b7-40c0-af25-396b62390d81'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:54.759651 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-e57cb279-34b7-40c0-af25-396b62390d81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd582c390822740d181e71c12c8bc1562', 'request_id': 'req-e57cb279-34b7-40c0-af25-396b62390d81'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:54.781441 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-e57cb279-34b7-40c0-af25-396b62390d81 admin admin] The token cache is being invalidate because project d582c390822740d181e71c12c8bc1562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:54.781718 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-e57cb279-34b7-40c0-af25-396b62390d81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:54.782397 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-e57cb279-34b7-40c0-af25-396b62390d81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:54.783960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 848/1696] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:54 2022] DELETE /identity/v3/projects/d582c390822740d181e71c12c8bc1562 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:54.788100 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11565e99-de6b-4c42-8b74-6e75dfe0870f service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:54.798698 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11565e99-de6b-4c42-8b74-6e75dfe0870f service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:54.856450 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682e945-0317-4057-a9f9-66b7656a7eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:54.857108 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682e945-0317-4057-a9f9-66b7656a7eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:54.857392 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682e945-0317-4057-a9f9-66b7656a7eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:54.857670 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682e945-0317-4057-a9f9-66b7656a7eed admin admin] PATH_INFO: `/v3/users/d1aee11e71454d74bec36a075c777dfa` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:54.858828 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682e945-0317-4057-a9f9-66b7656a7eed admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1aee11e71454d74bec36a075c777dfa)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:54.861010 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682e945-0317-4057-a9f9-66b7656a7eed admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:54.910626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-8682e945-0317-4057-a9f9-66b7656a7eed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1aee11e71454d74bec36a075c777dfa'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:54.918092 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-8682e945-0317-4057-a9f9-66b7656a7eed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1aee11e71454d74bec36a075c777dfa'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:54.930415 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-8682e945-0317-4057-a9f9-66b7656a7eed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1aee11e71454d74bec36a075c777dfa'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:54.931110 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-8682e945-0317-4057-a9f9-66b7656a7eed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1aee11e71454d74bec36a075c777dfa'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:54.931736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-8682e945-0317-4057-a9f9-66b7656a7eed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1aee11e71454d74bec36a075c777dfa'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:54.944596 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 849/1697] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:54 2022] DELETE /identity/v3/users/d1aee11e71454d74bec36a075c777dfa => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:55.232174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57cb279-34b7-40c0-af25-396b62390d81 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:55.234514 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-e57cb279-34b7-40c0-af25-396b62390d81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:55.251468 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57cb279-34b7-40c0-af25-396b62390d81 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:55.307487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:55.339341 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e8850-02d5-4ad7-ab41-ae412291d34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:55.340211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e8850-02d5-4ad7-ab41-ae412291d34b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:55.340855 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e8850-02d5-4ad7-ab41-ae412291d34b admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:55.341291 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e8850-02d5-4ad7-ab41-ae412291d34b admin admin] PATH_INFO: `/v3/projects/3ad7c30d41e241058ee1e9c3fe53b81e` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:55.342920 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e8850-02d5-4ad7-ab41-ae412291d34b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ad7c30d41e241058ee1e9c3fe53b81e)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:55.346918 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e8850-02d5-4ad7-ab41-ae412291d34b admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:55.415148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-916e8850-02d5-4ad7-ab41-ae412291d34b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ad7c30d41e241058ee1e9c3fe53b81e', 'request_id': 'req-916e8850-02d5-4ad7-ab41-ae412291d34b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:55.424197 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-916e8850-02d5-4ad7-ab41-ae412291d34b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ad7c30d41e241058ee1e9c3fe53b81e', 'request_id': 'req-916e8850-02d5-4ad7-ab41-ae412291d34b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:55.431578 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-916e8850-02d5-4ad7-ab41-ae412291d34b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ad7c30d41e241058ee1e9c3fe53b81e', 'request_id': 'req-916e8850-02d5-4ad7-ab41-ae412291d34b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:55.447496 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-916e8850-02d5-4ad7-ab41-ae412291d34b admin admin] The token cache is being invalidate because project 3ad7c30d41e241058ee1e9c3fe53b81e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:55.447796 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-916e8850-02d5-4ad7-ab41-ae412291d34b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:55.448689 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-916e8850-02d5-4ad7-ab41-ae412291d34b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:55.450314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 849/1698] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:55 2022] DELETE /identity/v3/projects/3ad7c30d41e241058ee1e9c3fe53b81e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:55.455094 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6db8d3-8dd5-46df-afd5-5e7fa396063a None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:55.455596 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6db8d3-8dd5-46df-afd5-5e7fa396063a None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:55.455690 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6db8d3-8dd5-46df-afd5-5e7fa396063a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:55.477555 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-9c6db8d3-8dd5-46df-afd5-5e7fa396063a None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:55.479047 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9c6db8d3-8dd5-46df-afd5-5e7fa396063a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:55.575082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 850/1699] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:55.583564 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a70568-97a1-4874-8acc-821c66ff39ea None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:55.583854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a70568-97a1-4874-8acc-821c66ff39ea None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:55.584134 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a70568-97a1-4874-8acc-821c66ff39ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:55.608195 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-84a70568-97a1-4874-8acc-821c66ff39ea None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:55.610067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-84a70568-97a1-4874-8acc-821c66ff39ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:55.616906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6db8d3-8dd5-46df-afd5-5e7fa396063a None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:55.618279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9c6db8d3-8dd5-46df-afd5-5e7fa396063a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:55.619732 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 850/1700] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:55.624369 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a70568-97a1-4874-8acc-821c66ff39ea None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:55.631706 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:55.634330 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a70568-97a1-4874-8acc-821c66ff39ea None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:55.641325 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377637a-1e4a-4064-918a-b773d1138aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:55.641944 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377637a-1e4a-4064-918a-b773d1138aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:55.642230 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377637a-1e4a-4064-918a-b773d1138aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:55.642511 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377637a-1e4a-4064-918a-b773d1138aa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:55.643357 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377637a-1e4a-4064-918a-b773d1138aa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:08:55.643712 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377637a-1e4a-4064-918a-b773d1138aa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:55.645501 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377637a-1e4a-4064-918a-b773d1138aa1 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:55.669271 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 851/1701] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:08:55.670402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377637a-1e4a-4064-918a-b773d1138aa1 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:55.685451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377637a-1e4a-4064-918a-b773d1138aa1 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:55.691994 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb3c15-7bad-4189-a974-afc5e190d917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:55.692735 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb3c15-7bad-4189-a974-afc5e190d917 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:55.693131 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb3c15-7bad-4189-a974-afc5e190d917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:55.693410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fb3c15-7bad-4189-a974-afc5e190d917 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:55.694111 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fb3c15-7bad-4189-a974-afc5e190d917 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:55.696032 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fb3c15-7bad-4189-a974-afc5e190d917 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:55.703992 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6db8d3-8dd5-46df-afd5-5e7fa396063a None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:55.708803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767a746-0fd2-45a1-8f92-3684d6c42655 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:55.709499 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767a746-0fd2-45a1-8f92-3684d6c42655 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:55.709793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767a746-0fd2-45a1-8f92-3684d6c42655 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:55.710061 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767a746-0fd2-45a1-8f92-3684d6c42655 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:55.712380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-8767a746-0fd2-45a1-8f92-3684d6c42655 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:55.744672 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 852/1702] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:55 2022] POST /identity/v3/projects => generated 387 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:55.750782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb3c15-7bad-4189-a974-afc5e190d917 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:55.800846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8767a746-0fd2-45a1-8f92-3684d6c42655 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:55.802725 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8767a746-0fd2-45a1-8f92-3684d6c42655 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:55.816054 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 851/1703] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:55 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:08:55.868096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fb3c15-7bad-4189-a974-afc5e190d917 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:55.878047 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd749a5d-e664-483b-b5ea-3a6e1c272ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:55.878885 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd749a5d-e664-483b-b5ea-3a6e1c272ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:55.879177 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd749a5d-e664-483b-b5ea-3a6e1c272ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:55.879557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd749a5d-e664-483b-b5ea-3a6e1c272ec3 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:55.880398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd749a5d-e664-483b-b5ea-3a6e1c272ec3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:55.882846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd749a5d-e664-483b-b5ea-3a6e1c272ec3 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:55.910178 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 853/1704] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:55 2022] POST /identity/v3/users => generated 294 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:55.911396 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767a746-0fd2-45a1-8f92-3684d6c42655 service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:55.923105 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767a746-0fd2-45a1-8f92-3684d6c42655 service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:55.928537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e6717-db43-44af-9e86-4289efefaab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:55.929297 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e6717-db43-44af-9e86-4289efefaab1 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:55.929460 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e6717-db43-44af-9e86-4289efefaab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:55.929737 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e6717-db43-44af-9e86-4289efefaab1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:55.930369 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e6717-db43-44af-9e86-4289efefaab1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:55.931947 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e6717-db43-44af-9e86-4289efefaab1 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:55.942740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 852/1705] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:55.946764 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd749a5d-e664-483b-b5ea-3a6e1c272ec3 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:55.958531 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd749a5d-e664-483b-b5ea-3a6e1c272ec3 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:55.962669 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1010d4-7d6c-4ef4-8b65-050db6daae0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:55.963282 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1010d4-7d6c-4ef4-8b65-050db6daae0a admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:55.963594 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1010d4-7d6c-4ef4-8b65-050db6daae0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:55.963894 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1010d4-7d6c-4ef4-8b65-050db6daae0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:55.964621 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1010d4-7d6c-4ef4-8b65-050db6daae0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:55.966325 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1010d4-7d6c-4ef4-8b65-050db6daae0a admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:55.980574 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 854/1706] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:55.982079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e6717-db43-44af-9e86-4289efefaab1 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:55.993404 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e6717-db43-44af-9e86-4289efefaab1 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:55.999656 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9213c18-c36f-47ba-a49c-47d57f0dcf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:56.000322 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9213c18-c36f-47ba-a49c-47d57f0dcf22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:56.000637 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9213c18-c36f-47ba-a49c-47d57f0dcf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:56.000917 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9213c18-c36f-47ba-a49c-47d57f0dcf22 admin admin] PATH_INFO: `/v3/projects/0f586c84083040d0a30b7639f35e2f54/users/02431ea9ed134f36b4eac39815d6fb81/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:56.019378 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9213c18-c36f-47ba-a49c-47d57f0dcf22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f586c84083040d0a30b7639f35e2f54, user_id=02431ea9ed134f36b4eac39815d6fb81, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:56.022427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9213c18-c36f-47ba-a49c-47d57f0dcf22 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:56.037731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 853/1707] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:55 2022] PUT /identity/v3/projects/0f586c84083040d0a30b7639f35e2f54/users/02431ea9ed134f36b4eac39815d6fb81/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:56.050404 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa06eb-2b60-49ee-8e20-fe96d8e95f36 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:56.050714 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa06eb-2b60-49ee-8e20-fe96d8e95f36 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:56.051089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa06eb-2b60-49ee-8e20-fe96d8e95f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:56.073567 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:56.084381 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-80aa06eb-2b60-49ee-8e20-fe96d8e95f36 None None] MFA Rules not processed for user `02431ea9ed134f36b4eac39815d6fb81`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:56.086340 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-80aa06eb-2b60-49ee-8e20-fe96d8e95f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:56.192161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 855/1708] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:08:56.196802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9213c18-c36f-47ba-a49c-47d57f0dcf22 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:56.272800 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9213c18-c36f-47ba-a49c-47d57f0dcf22 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:56.277498 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd52bc-7e81-4853-b905-4f6be69adb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:56.278091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd52bc-7e81-4853-b905-4f6be69adb88 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:56.278367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd52bc-7e81-4853-b905-4f6be69adb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:56.278641 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd52bc-7e81-4853-b905-4f6be69adb88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:56.279258 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd52bc-7e81-4853-b905-4f6be69adb88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:56.281662 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd52bc-7e81-4853-b905-4f6be69adb88 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:56.307974 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 854/1709] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:56 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:56.312614 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa06eb-2b60-49ee-8e20-fe96d8e95f36 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:56.323204 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa06eb-2b60-49ee-8e20-fe96d8e95f36 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:56.383118 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ecddf-a6d0-4248-b14d-7644937d3dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:56.383737 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ecddf-a6d0-4248-b14d-7644937d3dbe admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:56.384025 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ecddf-a6d0-4248-b14d-7644937d3dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:56.384321 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ecddf-a6d0-4248-b14d-7644937d3dbe admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:56.385023 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95ecddf-a6d0-4248-b14d-7644937d3dbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:56.387021 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95ecddf-a6d0-4248-b14d-7644937d3dbe admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:56.407616 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 856/1710] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:56 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:08:56.411587 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd52bc-7e81-4853-b905-4f6be69adb88 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:56.418797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:56.424412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd52bc-7e81-4853-b905-4f6be69adb88 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:56.430246 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbd71c-83fe-489a-a1ae-fd6eee79b3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:56.430861 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbd71c-83fe-489a-a1ae-fd6eee79b3a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:56.431145 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbd71c-83fe-489a-a1ae-fd6eee79b3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:56.431424 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbd71c-83fe-489a-a1ae-fd6eee79b3a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:56.432066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbd71c-83fe-489a-a1ae-fd6eee79b3a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:56.433685 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbd71c-83fe-489a-a1ae-fd6eee79b3a9 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:56.445078 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 855/1711] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:56.449502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ecddf-a6d0-4248-b14d-7644937d3dbe admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:56.461143 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ecddf-a6d0-4248-b14d-7644937d3dbe admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:56.465875 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b329db2-e16c-430a-9e4b-eefe5d694265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:56.466561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b329db2-e16c-430a-9e4b-eefe5d694265 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:56.466853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b329db2-e16c-430a-9e4b-eefe5d694265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:56.467135 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b329db2-e16c-430a-9e4b-eefe5d694265 admin admin] PATH_INFO: `/v3/projects/54f54a9c369c4fb2895a31f5239dde5a/users/b20a431af8f342cd99e5825bb5859627/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:56.493830 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b329db2-e16c-430a-9e4b-eefe5d694265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54f54a9c369c4fb2895a31f5239dde5a, user_id=b20a431af8f342cd99e5825bb5859627, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:56.499677 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b329db2-e16c-430a-9e4b-eefe5d694265 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:56.524611 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbd71c-83fe-489a-a1ae-fd6eee79b3a9 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:56.525139 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 857/1712] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:56 2022] PUT /identity/v3/projects/54f54a9c369c4fb2895a31f5239dde5a/users/b20a431af8f342cd99e5825bb5859627/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:56.535725 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbd71c-83fe-489a-a1ae-fd6eee79b3a9 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:56.627900 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b65047-044f-4d0a-a981-c7bad9a1c2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:56.628658 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b65047-044f-4d0a-a981-c7bad9a1c2a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:56.629009 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b65047-044f-4d0a-a981-c7bad9a1c2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:56.629379 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b65047-044f-4d0a-a981-c7bad9a1c2a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:56.630138 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b65047-044f-4d0a-a981-c7bad9a1c2a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:56.631802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b65047-044f-4d0a-a981-c7bad9a1c2a8 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:56.643096 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 856/1713] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:08:56.647328 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b329db2-e16c-430a-9e4b-eefe5d694265 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:56.659075 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b329db2-e16c-430a-9e4b-eefe5d694265 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:56.664173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4464ec-835f-4912-90d8-dd866c9257ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:56.664906 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4464ec-835f-4912-90d8-dd866c9257ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:56.665194 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4464ec-835f-4912-90d8-dd866c9257ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:56.665463 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4464ec-835f-4912-90d8-dd866c9257ba admin admin] PATH_INFO: `/v3/projects/54f54a9c369c4fb2895a31f5239dde5a/users/b20a431af8f342cd99e5825bb5859627/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:56.667497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4464ec-835f-4912-90d8-dd866c9257ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=54f54a9c369c4fb2895a31f5239dde5a, user_id=b20a431af8f342cd99e5825bb5859627, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:56.670192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4464ec-835f-4912-90d8-dd866c9257ba admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:56.684610 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 858/1714] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:56 2022] PUT /identity/v3/projects/54f54a9c369c4fb2895a31f5239dde5a/users/b20a431af8f342cd99e5825bb5859627/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:08:56.697392 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cadb5-fe18-49ce-ad97-a396987b420a None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:56.697653 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cadb5-fe18-49ce-ad97-a396987b420a None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:56.697921 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cadb5-fe18-49ce-ad97-a396987b420a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:56.734719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-c82cadb5-fe18-49ce-ad97-a396987b420a None None] MFA Rules not processed for user `b20a431af8f342cd99e5825bb5859627`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:56.736002 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-c82cadb5-fe18-49ce-ad97-a396987b420a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:56.845814 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 857/1715] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:56 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:56.857388 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4464ec-835f-4912-90d8-dd866c9257ba admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:56.941440 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4464ec-835f-4912-90d8-dd866c9257ba admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:56.945928 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0a96a-9470-4f1a-9fe6-7a17e956098f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:56.946734 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0a96a-9470-4f1a-9fe6-7a17e956098f service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:56.947236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0a96a-9470-4f1a-9fe6-7a17e956098f service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:56.947756 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0a96a-9470-4f1a-9fe6-7a17e956098f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:56.955787 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:56.960973 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0a96a-9470-4f1a-9fe6-7a17e956098f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:56.963286 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0a96a-9470-4f1a-9fe6-7a17e956098f service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:56.977591 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 859/1716] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:56 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:57.027300 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cadb5-fe18-49ce-ad97-a396987b420a None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:57.133141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cadb5-fe18-49ce-ad97-a396987b420a None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:57.138439 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7c121-caa7-4f1c-9a6c-921be42427bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:57.139371 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7c121-caa7-4f1c-9a6c-921be42427bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:57.140235 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7c121-caa7-4f1c-9a6c-921be42427bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:57.140747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7c121-caa7-4f1c-9a6c-921be42427bb admin admin] PATH_INFO: `/v3/users/02431ea9ed134f36b4eac39815d6fb81` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:57.152511 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f7c121-caa7-4f1c-9a6c-921be42427bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=02431ea9ed134f36b4eac39815d6fb81)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:57.152511 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f7c121-caa7-4f1c-9a6c-921be42427bb admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:57.163416 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:57.220421 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c9f7c121-caa7-4f1c-9a6c-921be42427bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02431ea9ed134f36b4eac39815d6fb81'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:57.233615 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c9f7c121-caa7-4f1c-9a6c-921be42427bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02431ea9ed134f36b4eac39815d6fb81'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:57.250076 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c9f7c121-caa7-4f1c-9a6c-921be42427bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02431ea9ed134f36b4eac39815d6fb81'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:57.250933 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c9f7c121-caa7-4f1c-9a6c-921be42427bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02431ea9ed134f36b4eac39815d6fb81'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:57.251609 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c9f7c121-caa7-4f1c-9a6c-921be42427bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02431ea9ed134f36b4eac39815d6fb81'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:57.266879 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 858/1717] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:57 2022] DELETE /identity/v3/users/02431ea9ed134f36b4eac39815d6fb81 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:57.271716 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d7e1a-9c0e-424e-9a9c-1deebb15a054 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:57.272008 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d7e1a-9c0e-424e-9a9c-1deebb15a054 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:57.272278 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d7e1a-9c0e-424e-9a9c-1deebb15a054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:57.288870 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7c121-caa7-4f1c-9a6c-921be42427bb admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:57.290275 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-c9f7c121-caa7-4f1c-9a6c-921be42427bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:57.294641 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-314d7e1a-9c0e-424e-9a9c-1deebb15a054 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:57.297260 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-314d7e1a-9c0e-424e-9a9c-1deebb15a054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:57.380430 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 860/1718] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:08:57.381055 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7c121-caa7-4f1c-9a6c-921be42427bb admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:57.385345 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce8cd1-d88e-423a-a8fb-ac66395b101b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:57.386033 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce8cd1-d88e-423a-a8fb-ac66395b101b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:57.387218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce8cd1-d88e-423a-a8fb-ac66395b101b admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:57.387218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce8cd1-d88e-423a-a8fb-ac66395b101b admin admin] PATH_INFO: `/v3/users/741fc52ad5a742cc89eb6c8253cb7f25` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:57.387807 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce8cd1-d88e-423a-a8fb-ac66395b101b admin admin] RBAC: Authorizing `identity:delete_user(user_id=741fc52ad5a742cc89eb6c8253cb7f25)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:57.388742 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d7e1a-9c0e-424e-9a9c-1deebb15a054 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:57.390402 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce8cd1-d88e-423a-a8fb-ac66395b101b admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:57.391344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-314d7e1a-9c0e-424e-9a9c-1deebb15a054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:57.460201 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c1ce8cd1-d88e-423a-a8fb-ac66395b101b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '741fc52ad5a742cc89eb6c8253cb7f25'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:57.465568 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d7e1a-9c0e-424e-9a9c-1deebb15a054 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:57.469983 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c1ce8cd1-d88e-423a-a8fb-ac66395b101b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '741fc52ad5a742cc89eb6c8253cb7f25'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:57.470632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7b608-d833-4bf0-b708-4202ecae9e5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:57.471195 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7b608-d833-4bf0-b708-4202ecae9e5e service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:57.471477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7b608-d833-4bf0-b708-4202ecae9e5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:57.471773 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7b608-d833-4bf0-b708-4202ecae9e5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:57.485588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c1ce8cd1-d88e-423a-a8fb-ac66395b101b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '741fc52ad5a742cc89eb6c8253cb7f25'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:57.485988 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a7b608-d833-4bf0-b708-4202ecae9e5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:57.486491 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c1ce8cd1-d88e-423a-a8fb-ac66395b101b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '741fc52ad5a742cc89eb6c8253cb7f25'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:57.487207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c1ce8cd1-d88e-423a-a8fb-ac66395b101b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '741fc52ad5a742cc89eb6c8253cb7f25'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:57.487699 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a7b608-d833-4bf0-b708-4202ecae9e5e service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:57.504987 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 859/1719] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:57 2022] DELETE /identity/v3/users/741fc52ad5a742cc89eb6c8253cb7f25 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:08:57.505345 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 861/1720] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:57.508583 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e216b-1b0d-48f2-9774-20ceb8f0c2a0 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:57.508898 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e216b-1b0d-48f2-9774-20ceb8f0c2a0 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:57.509166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e216b-1b0d-48f2-9774-20ceb8f0c2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:57.533518 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-5b9e216b-1b0d-48f2-9774-20ceb8f0c2a0 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:57.534800 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5b9e216b-1b0d-48f2-9774-20ceb8f0c2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:57.544535 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:57.605177 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 860/1721] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:57.615682 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7b608-d833-4bf0-b708-4202ecae9e5e service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:57.617585 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c3a7b608-d833-4bf0-b708-4202ecae9e5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:57.632148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7b608-d833-4bf0-b708-4202ecae9e5e service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:57.706614 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c92567-572e-411e-b589-4617a27c5f8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:57.707334 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c92567-572e-411e-b589-4617a27c5f8b service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:57.707914 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c92567-572e-411e-b589-4617a27c5f8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:57.708257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c92567-572e-411e-b589-4617a27c5f8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:57.718023 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c92567-572e-411e-b589-4617a27c5f8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:57.719546 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c92567-572e-411e-b589-4617a27c5f8b service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:57.729638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 862/1722] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:08:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:08:57.953632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e216b-1b0d-48f2-9774-20ceb8f0c2a0 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:57.955190 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5b9e216b-1b0d-48f2-9774-20ceb8f0c2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:57.969549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e216b-1b0d-48f2-9774-20ceb8f0c2a0 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:57.977208 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1eb889-986f-4574-aae3-c070a1e7c56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:57.977998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1eb889-986f-4574-aae3-c070a1e7c56d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:57.978383 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1eb889-986f-4574-aae3-c070a1e7c56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:57.978748 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1eb889-986f-4574-aae3-c070a1e7c56d admin admin] PATH_INFO: `/v3/projects/0f586c84083040d0a30b7639f35e2f54` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:57.979989 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1eb889-986f-4574-aae3-c070a1e7c56d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f586c84083040d0a30b7639f35e2f54)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:57.982143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1eb889-986f-4574-aae3-c070a1e7c56d admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:58.037820 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-2f1eb889-986f-4574-aae3-c070a1e7c56d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f586c84083040d0a30b7639f35e2f54', 'request_id': 'req-2f1eb889-986f-4574-aae3-c070a1e7c56d'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.044257 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-2f1eb889-986f-4574-aae3-c070a1e7c56d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f586c84083040d0a30b7639f35e2f54', 'request_id': 'req-2f1eb889-986f-4574-aae3-c070a1e7c56d'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.053769 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-2f1eb889-986f-4574-aae3-c070a1e7c56d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f586c84083040d0a30b7639f35e2f54', 'request_id': 'req-2f1eb889-986f-4574-aae3-c070a1e7c56d'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.068171 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-2f1eb889-986f-4574-aae3-c070a1e7c56d admin admin] The token cache is being invalidate because project 0f586c84083040d0a30b7639f35e2f54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:58.068560 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-2f1eb889-986f-4574-aae3-c070a1e7c56d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.069228 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-2f1eb889-986f-4574-aae3-c070a1e7c56d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.071033 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 861/1723] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:57 2022] DELETE /identity/v3/projects/0f586c84083040d0a30b7639f35e2f54 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:58.074911 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c92567-572e-411e-b589-4617a27c5f8b service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:58.086616 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c92567-572e-411e-b589-4617a27c5f8b service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:58.138695 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:58.159266 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a57de6-d121-4caa-aeeb-0fabe7c778ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:58.159937 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a57de6-d121-4caa-aeeb-0fabe7c778ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:58.160219 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a57de6-d121-4caa-aeeb-0fabe7c778ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:58.160512 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a57de6-d121-4caa-aeeb-0fabe7c778ab admin admin] PATH_INFO: `/v3/users/b20a431af8f342cd99e5825bb5859627` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:58.165253 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a57de6-d121-4caa-aeeb-0fabe7c778ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=b20a431af8f342cd99e5825bb5859627)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:58.168708 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a57de6-d121-4caa-aeeb-0fabe7c778ab admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:58.224896 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1eb889-986f-4574-aae3-c070a1e7c56d admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:58.227087 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-2f1eb889-986f-4574-aae3-c070a1e7c56d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:58.236926 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-10a57de6-d121-4caa-aeeb-0fabe7c778ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b20a431af8f342cd99e5825bb5859627'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.237846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-10a57de6-d121-4caa-aeeb-0fabe7c778ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b20a431af8f342cd99e5825bb5859627'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.247936 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-10a57de6-d121-4caa-aeeb-0fabe7c778ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b20a431af8f342cd99e5825bb5859627'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.248809 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1eb889-986f-4574-aae3-c070a1e7c56d admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:58.257144 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddea4ce-ef92-4e32-bb44-2638e70efe42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:58.258062 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddea4ce-ef92-4e32-bb44-2638e70efe42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:58.258349 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddea4ce-ef92-4e32-bb44-2638e70efe42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:58.258629 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddea4ce-ef92-4e32-bb44-2638e70efe42 admin admin] PATH_INFO: `/v3/projects/a572adef60f9491e89d0cc74bbde26d2` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:58.260369 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddea4ce-ef92-4e32-bb44-2638e70efe42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a572adef60f9491e89d0cc74bbde26d2)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:58.270212 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-10a57de6-d121-4caa-aeeb-0fabe7c778ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b20a431af8f342cd99e5825bb5859627'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.270519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddea4ce-ef92-4e32-bb44-2638e70efe42 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:58.290276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-10a57de6-d121-4caa-aeeb-0fabe7c778ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b20a431af8f342cd99e5825bb5859627'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.302489 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 863/1724] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:58 2022] DELETE /identity/v3/users/b20a431af8f342cd99e5825bb5859627 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:58.333272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-5ddea4ce-ef92-4e32-bb44-2638e70efe42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a572adef60f9491e89d0cc74bbde26d2', 'request_id': 'req-5ddea4ce-ef92-4e32-bb44-2638e70efe42'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.339644 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-5ddea4ce-ef92-4e32-bb44-2638e70efe42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a572adef60f9491e89d0cc74bbde26d2', 'request_id': 'req-5ddea4ce-ef92-4e32-bb44-2638e70efe42'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.347848 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-5ddea4ce-ef92-4e32-bb44-2638e70efe42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a572adef60f9491e89d0cc74bbde26d2', 'request_id': 'req-5ddea4ce-ef92-4e32-bb44-2638e70efe42'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.360089 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-5ddea4ce-ef92-4e32-bb44-2638e70efe42 admin admin] The token cache is being invalidate because project a572adef60f9491e89d0cc74bbde26d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:58.360358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-5ddea4ce-ef92-4e32-bb44-2638e70efe42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.360991 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-5ddea4ce-ef92-4e32-bb44-2638e70efe42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.362523 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 862/1725] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:58 2022] DELETE /identity/v3/projects/a572adef60f9491e89d0cc74bbde26d2 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:58.366866 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a57de6-d121-4caa-aeeb-0fabe7c778ab admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:58.368633 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-10a57de6-d121-4caa-aeeb-0fabe7c778ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:58.442343 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a57de6-d121-4caa-aeeb-0fabe7c778ab admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:58.507727 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90428407-72d1-47b6-a649-1831d073cbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:58.508471 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90428407-72d1-47b6-a649-1831d073cbbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:58.508801 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90428407-72d1-47b6-a649-1831d073cbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:58.509252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90428407-72d1-47b6-a649-1831d073cbbd admin admin] PATH_INFO: `/v3/users/d22b209958844340a5a3538a2fbbfe01` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:58.510369 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90428407-72d1-47b6-a649-1831d073cbbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d22b209958844340a5a3538a2fbbfe01)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:58.512525 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90428407-72d1-47b6-a649-1831d073cbbd admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:58.519872 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:58.576571 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-90428407-72d1-47b6-a649-1831d073cbbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd22b209958844340a5a3538a2fbbfe01'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.578337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-90428407-72d1-47b6-a649-1831d073cbbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd22b209958844340a5a3538a2fbbfe01'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.587023 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-90428407-72d1-47b6-a649-1831d073cbbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd22b209958844340a5a3538a2fbbfe01'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.608132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-90428407-72d1-47b6-a649-1831d073cbbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd22b209958844340a5a3538a2fbbfe01'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.616979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-90428407-72d1-47b6-a649-1831d073cbbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd22b209958844340a5a3538a2fbbfe01'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.623613 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 864/1726] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:08:58 2022] DELETE /identity/v3/users/d22b209958844340a5a3538a2fbbfe01 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:58.663648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddea4ce-ef92-4e32-bb44-2638e70efe42 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:58.665339 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5ddea4ce-ef92-4e32-bb44-2638e70efe42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:58.679904 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddea4ce-ef92-4e32-bb44-2638e70efe42 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:58.702870 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:58.747695 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988cfffc-c244-4c77-9328-ddf853c25004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:58.748512 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cfffc-c244-4c77-9328-ddf853c25004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:58.748697 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cfffc-c244-4c77-9328-ddf853c25004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:58.748953 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cfffc-c244-4c77-9328-ddf853c25004 admin admin] PATH_INFO: `/v3/projects/54f54a9c369c4fb2895a31f5239dde5a` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:58.751134 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988cfffc-c244-4c77-9328-ddf853c25004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54f54a9c369c4fb2895a31f5239dde5a)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:58.753307 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988cfffc-c244-4c77-9328-ddf853c25004 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:58.803895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-988cfffc-c244-4c77-9328-ddf853c25004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54f54a9c369c4fb2895a31f5239dde5a', 'request_id': 'req-988cfffc-c244-4c77-9328-ddf853c25004'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.810286 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-988cfffc-c244-4c77-9328-ddf853c25004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54f54a9c369c4fb2895a31f5239dde5a', 'request_id': 'req-988cfffc-c244-4c77-9328-ddf853c25004'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.818820 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-988cfffc-c244-4c77-9328-ddf853c25004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54f54a9c369c4fb2895a31f5239dde5a', 'request_id': 'req-988cfffc-c244-4c77-9328-ddf853c25004'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.835584 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-988cfffc-c244-4c77-9328-ddf853c25004 admin admin] The token cache is being invalidate because project 54f54a9c369c4fb2895a31f5239dde5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:58.835876 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-988cfffc-c244-4c77-9328-ddf853c25004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.837001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-988cfffc-c244-4c77-9328-ddf853c25004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:58.838558 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 863/1727] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:58 2022] DELETE /identity/v3/projects/54f54a9c369c4fb2895a31f5239dde5a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:58.844205 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cd575-362e-419e-94f1-f23fde808396 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:58.844515 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cd575-362e-419e-94f1-f23fde808396 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:58.844810 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cd575-362e-419e-94f1-f23fde808396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:58.865211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-982cd575-362e-419e-94f1-f23fde808396 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:58.866425 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-982cd575-362e-419e-94f1-f23fde808396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:58.946798 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988cfffc-c244-4c77-9328-ddf853c25004 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:58.948830 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-988cfffc-c244-4c77-9328-ddf853c25004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:58.959080 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 865/1728] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:08:58.967433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e2d9e-f7b2-475a-a145-a913588d5af3 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:58.968576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e2d9e-f7b2-475a-a145-a913588d5af3 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:58.968866 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e2d9e-f7b2-475a-a145-a913588d5af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:58.993064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-2f1e2d9e-f7b2-475a-a145-a913588d5af3 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:58.995008 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2f1e2d9e-f7b2-475a-a145-a913588d5af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:59.008408 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 866/1729] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:59.008711 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e2d9e-f7b2-475a-a145-a913588d5af3 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:59.018566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e2d9e-f7b2-475a-a145-a913588d5af3 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:59.023056 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bdfb3-ee86-47a0-86e1-6602da6fbec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:59.023668 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bdfb3-ee86-47a0-86e1-6602da6fbec2 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:59.023955 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bdfb3-ee86-47a0-86e1-6602da6fbec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:59.024232 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bdfb3-ee86-47a0-86e1-6602da6fbec2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:59.024976 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0bdfb3-ee86-47a0-86e1-6602da6fbec2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:08:59.025281 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0bdfb3-ee86-47a0-86e1-6602da6fbec2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:59.027448 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0bdfb3-ee86-47a0-86e1-6602da6fbec2 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:59.044253 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 867/1730] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:08:59.045284 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bdfb3-ee86-47a0-86e1-6602da6fbec2 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:59.047447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988cfffc-c244-4c77-9328-ddf853c25004 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:59.053079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c1e07-57e5-433d-ad3a-eb363351f55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:59.053809 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c1e07-57e5-433d-ad3a-eb363351f55c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:59.054100 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c1e07-57e5-433d-ad3a-eb363351f55c admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:59.054441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c1e07-57e5-433d-ad3a-eb363351f55c admin admin] PATH_INFO: `/v3/projects/486be0966e3d42d192a16f8da75cb319` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:59.055720 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c1e07-57e5-433d-ad3a-eb363351f55c admin admin] RBAC: Authorizing `identity:delete_project(project_id=486be0966e3d42d192a16f8da75cb319)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:59.056697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bdfb3-ee86-47a0-86e1-6602da6fbec2 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:59.057670 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c1e07-57e5-433d-ad3a-eb363351f55c admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:59.062408 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ed1ab-e5fb-410f-9a71-af2c8cce541c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:59.063202 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ed1ab-e5fb-410f-9a71-af2c8cce541c admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:59.063927 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ed1ab-e5fb-410f-9a71-af2c8cce541c admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:59.063927 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646ed1ab-e5fb-410f-9a71-af2c8cce541c admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:59.064672 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ed1ab-e5fb-410f-9a71-af2c8cce541c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:59.066680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646ed1ab-e5fb-410f-9a71-af2c8cce541c admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:59.102233 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 868/1731] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:59 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:08:59.103239 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ed1ab-e5fb-410f-9a71-af2c8cce541c admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:59.147497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-171c1e07-57e5-433d-ad3a-eb363351f55c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '486be0966e3d42d192a16f8da75cb319', 'request_id': 'req-171c1e07-57e5-433d-ad3a-eb363351f55c'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:59.149552 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:59.155046 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-171c1e07-57e5-433d-ad3a-eb363351f55c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '486be0966e3d42d192a16f8da75cb319', 'request_id': 'req-171c1e07-57e5-433d-ad3a-eb363351f55c'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:59.160231 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:59.164993 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-171c1e07-57e5-433d-ad3a-eb363351f55c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '486be0966e3d42d192a16f8da75cb319', 'request_id': 'req-171c1e07-57e5-433d-ad3a-eb363351f55c'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:59.176975 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-171c1e07-57e5-433d-ad3a-eb363351f55c admin admin] The token cache is being invalidate because project 486be0966e3d42d192a16f8da75cb319 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:08:59.177274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-171c1e07-57e5-433d-ad3a-eb363351f55c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:59.177919 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-171c1e07-57e5-433d-ad3a-eb363351f55c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:08:59.179275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646ed1ab-e5fb-410f-9a71-af2c8cce541c admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:59.179439 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 864/1732] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:08:58 2022] DELETE /identity/v3/projects/486be0966e3d42d192a16f8da75cb319 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:08:59.184335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9470e7-5406-4226-a918-01fe363cfe18 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:59.184478 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9470e7-5406-4226-a918-01fe363cfe18 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:59.184784 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9470e7-5406-4226-a918-01fe363cfe18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:59.205327 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-2f9470e7-5406-4226-a918-01fe363cfe18 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:59.206582 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-2f9470e7-5406-4226-a918-01fe363cfe18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:59.238606 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793d327-3f2b-45f9-a064-b0f0aa3b2de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:59.239216 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793d327-3f2b-45f9-a064-b0f0aa3b2de2 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:59.239502 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793d327-3f2b-45f9-a064-b0f0aa3b2de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:59.239778 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793d327-3f2b-45f9-a064-b0f0aa3b2de2 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:59.240529 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793d327-3f2b-45f9-a064-b0f0aa3b2de2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:59.242330 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793d327-3f2b-45f9-a064-b0f0aa3b2de2 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:59.265296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 869/1733] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:59 2022] POST /identity/v3/users => generated 284 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:08:59.269468 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793d327-3f2b-45f9-a064-b0f0aa3b2de2 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:59.271215 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 865/1734] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:08:59.271340 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b793d327-3f2b-45f9-a064-b0f0aa3b2de2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:59.278835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a0d84-ac44-40db-8040-9a5103932963 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:59.279123 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a0d84-ac44-40db-8040-9a5103932963 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:59.279400 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a0d84-ac44-40db-8040-9a5103932963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:59.285238 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793d327-3f2b-45f9-a064-b0f0aa3b2de2 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:59.290914 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442bc15-6a47-44cb-8f17-9810f7f2889d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:59.291549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442bc15-6a47-44cb-8f17-9810f7f2889d admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:59.291833 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442bc15-6a47-44cb-8f17-9810f7f2889d admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:59.292112 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442bc15-6a47-44cb-8f17-9810f7f2889d admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:59.292791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442bc15-6a47-44cb-8f17-9810f7f2889d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:59.294380 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442bc15-6a47-44cb-8f17-9810f7f2889d admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:59.298832 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-597a0d84-ac44-40db-8040-9a5103932963 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:59.300018 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-597a0d84-ac44-40db-8040-9a5103932963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:59.304706 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 870/1735] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:08:59.307842 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 866/1736] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:08:59.308725 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442bc15-6a47-44cb-8f17-9810f7f2889d admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:59.311895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a0d84-ac44-40db-8040-9a5103932963 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:59.321072 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442bc15-6a47-44cb-8f17-9810f7f2889d admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:59.321508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a0d84-ac44-40db-8040-9a5103932963 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:59.326549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f772d-3de7-4240-a53b-fe143067581f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:59.326996 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79fd799-aa70-48e6-9a53-bd1d57037510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:59.328118 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79fd799-aa70-48e6-9a53-bd1d57037510 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:59.328394 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79fd799-aa70-48e6-9a53-bd1d57037510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:59.328543 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f772d-3de7-4240-a53b-fe143067581f admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:59.328696 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79fd799-aa70-48e6-9a53-bd1d57037510 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:59.328863 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f772d-3de7-4240-a53b-fe143067581f admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:59.329216 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f772d-3de7-4240-a53b-fe143067581f admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:59.329316 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79fd799-aa70-48e6-9a53-bd1d57037510 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:08:59.329623 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79fd799-aa70-48e6-9a53-bd1d57037510 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:59.329998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793f772d-3de7-4240-a53b-fe143067581f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:59.331212 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79fd799-aa70-48e6-9a53-bd1d57037510 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:59.331581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793f772d-3de7-4240-a53b-fe143067581f admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:59.344615 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 867/1737] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:08:59.349690 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 871/1738] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:08:59.349832 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79fd799-aa70-48e6-9a53-bd1d57037510 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:59.351051 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f772d-3de7-4240-a53b-fe143067581f admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:59.359197 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79fd799-aa70-48e6-9a53-bd1d57037510 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:59.362539 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f772d-3de7-4240-a53b-fe143067581f admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:59.364581 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d757b4-0c9b-4e7f-bb53-e4938b3b3f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:59.365466 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d757b4-0c9b-4e7f-bb53-e4938b3b3f4d admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:59.365961 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d757b4-0c9b-4e7f-bb53-e4938b3b3f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:59.366496 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d757b4-0c9b-4e7f-bb53-e4938b3b3f4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:59.367368 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d757b4-0c9b-4e7f-bb53-e4938b3b3f4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:59.367579 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df438030-f329-4b9f-be7e-1a4148306369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:59.368235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df438030-f329-4b9f-be7e-1a4148306369 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:59.368538 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df438030-f329-4b9f-be7e-1a4148306369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:59.368823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df438030-f329-4b9f-be7e-1a4148306369 admin admin] PATH_INFO: `/v3/projects/3587e476aa664be0bc026f290377e7d7/users/cd39b19e427145a08929287b48871663/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:59.370485 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d757b4-0c9b-4e7f-bb53-e4938b3b3f4d admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:59.385023 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df438030-f329-4b9f-be7e-1a4148306369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3587e476aa664be0bc026f290377e7d7, user_id=cd39b19e427145a08929287b48871663, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:59.388158 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df438030-f329-4b9f-be7e-1a4148306369 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:59.404765 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 872/1739] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:59 2022] PUT /identity/v3/projects/3587e476aa664be0bc026f290377e7d7/users/cd39b19e427145a08929287b48871663/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:08:59.413865 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 868/1740] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:59 2022] POST /identity/v3/projects => generated 409 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:08:59.417180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df438030-f329-4b9f-be7e-1a4148306369 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:59.422084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253c090-4abf-4b41-acc2-ddf1316d6c9f None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:59.422380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253c090-4abf-4b41-acc2-ddf1316d6c9f None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:59.422667 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e253c090-4abf-4b41-acc2-ddf1316d6c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:59.455490 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-e253c090-4abf-4b41-acc2-ddf1316d6c9f None None] MFA Rules not processed for user `cd39b19e427145a08929287b48871663`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:59.456797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e253c090-4abf-4b41-acc2-ddf1316d6c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:59.487996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df438030-f329-4b9f-be7e-1a4148306369 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:59.536958 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 869/1741] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:08:59.541146 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e253c090-4abf-4b41-acc2-ddf1316d6c9f None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:59.549421 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:59.570806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23541f66-4912-4ea4-a01b-99bc67f8fefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:59.571537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23541f66-4912-4ea4-a01b-99bc67f8fefe admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:59.571794 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23541f66-4912-4ea4-a01b-99bc67f8fefe admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:59.572079 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23541f66-4912-4ea4-a01b-99bc67f8fefe admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:59.572821 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23541f66-4912-4ea4-a01b-99bc67f8fefe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:59.574883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23541f66-4912-4ea4-a01b-99bc67f8fefe admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:59.577077 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:59.604538 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 873/1742] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:59 2022] POST /identity/v3/users => generated 305 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:08:59.607471 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23541f66-4912-4ea4-a01b-99bc67f8fefe admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:59.619417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23541f66-4912-4ea4-a01b-99bc67f8fefe admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:59.625332 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b9c17-c784-4a08-aa75-06649fdc4b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:59.625852 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b9c17-c784-4a08-aa75-06649fdc4b84 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:59.626134 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b9c17-c784-4a08-aa75-06649fdc4b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:59.626520 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b9c17-c784-4a08-aa75-06649fdc4b84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:59.627155 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4b9c17-c784-4a08-aa75-06649fdc4b84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:59.628769 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4b9c17-c784-4a08-aa75-06649fdc4b84 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:59.636365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e253c090-4abf-4b41-acc2-ddf1316d6c9f None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:59.639333 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 874/1743] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:59.642329 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266640d-9a4a-47d7-8f74-96ea3cfffb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:59.642957 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266640d-9a4a-47d7-8f74-96ea3cfffb3f admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:59.643240 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266640d-9a4a-47d7-8f74-96ea3cfffb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:59.643518 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266640d-9a4a-47d7-8f74-96ea3cfffb3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:59.644093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b9c17-c784-4a08-aa75-06649fdc4b84 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:59.644203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266640d-9a4a-47d7-8f74-96ea3cfffb3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:59.646087 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266640d-9a4a-47d7-8f74-96ea3cfffb3f admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:59.660723 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b9c17-c784-4a08-aa75-06649fdc4b84 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:59.665220 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85689a1f-5ab2-4bf6-a37f-8ebc2ffda05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:59.665829 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85689a1f-5ab2-4bf6-a37f-8ebc2ffda05a admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:59.666116 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85689a1f-5ab2-4bf6-a37f-8ebc2ffda05a admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:59.666437 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85689a1f-5ab2-4bf6-a37f-8ebc2ffda05a admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:59.667073 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85689a1f-5ab2-4bf6-a37f-8ebc2ffda05a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:59.668727 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85689a1f-5ab2-4bf6-a37f-8ebc2ffda05a admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:59.684533 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 875/1744] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:59.684671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 870/1745] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:59 2022] POST /identity/v3/projects => generated 365 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:08:59.687208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85689a1f-5ab2-4bf6-a37f-8ebc2ffda05a admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:59.687986 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266640d-9a4a-47d7-8f74-96ea3cfffb3f admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:59.697126 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85689a1f-5ab2-4bf6-a37f-8ebc2ffda05a admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:59.753705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266640d-9a4a-47d7-8f74-96ea3cfffb3f admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:59.755488 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9e280-658d-48dc-9a41-f03285cb3008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:59.756159 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9e280-658d-48dc-9a41-f03285cb3008 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:59.756443 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9e280-658d-48dc-9a41-f03285cb3008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:59.756766 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9e280-658d-48dc-9a41-f03285cb3008 admin admin] PATH_INFO: `/v3/projects/618580c833ef499e9d4bec5c868c2354/users/746acc5102f84af9af6e513e8e902910/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:59.757907 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d732389-59a9-41e3-9601-ee58adafa61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:59.758505 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d732389-59a9-41e3-9601-ee58adafa61b admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:59.758812 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d732389-59a9-41e3-9601-ee58adafa61b admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:59.759079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d732389-59a9-41e3-9601-ee58adafa61b admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:59.759702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d732389-59a9-41e3-9601-ee58adafa61b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:59.761656 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d732389-59a9-41e3-9601-ee58adafa61b admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:59.772166 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9e280-658d-48dc-9a41-f03285cb3008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=618580c833ef499e9d4bec5c868c2354, user_id=746acc5102f84af9af6e513e8e902910, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:59.774929 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9e280-658d-48dc-9a41-f03285cb3008 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:59.787978 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 871/1746] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:59 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:08:59.788338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 876/1747] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:59 2022] PUT /identity/v3/projects/618580c833ef499e9d4bec5c868c2354/users/746acc5102f84af9af6e513e8e902910/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:08:59.789550 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d732389-59a9-41e3-9601-ee58adafa61b admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:59.798631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33fa63a-7d32-40bb-b917-4f3352b5c478 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:59.799038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33fa63a-7d32-40bb-b917-4f3352b5c478 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:59.799495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33fa63a-7d32-40bb-b917-4f3352b5c478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:59.799693 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d732389-59a9-41e3-9601-ee58adafa61b admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:59.830183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-e33fa63a-7d32-40bb-b917-4f3352b5c478 None None] MFA Rules not processed for user `746acc5102f84af9af6e513e8e902910`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:08:59.831420 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-e33fa63a-7d32-40bb-b917-4f3352b5c478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:08:59.859885 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d64a13-5d4b-4df9-91e8-83684b199c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:59.860514 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d64a13-5d4b-4df9-91e8-83684b199c1e admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:59.860825 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d64a13-5d4b-4df9-91e8-83684b199c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:59.861102 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d64a13-5d4b-4df9-91e8-83684b199c1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:59.861744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d64a13-5d4b-4df9-91e8-83684b199c1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:59.863335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d64a13-5d4b-4df9-91e8-83684b199c1e admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:59.873120 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 872/1748] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:08:59.877443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d64a13-5d4b-4df9-91e8-83684b199c1e admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:59.888114 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d64a13-5d4b-4df9-91e8-83684b199c1e admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:59.893455 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aadb0a-d371-450b-8d8d-4caa20ca70a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:59.894228 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aadb0a-d371-450b-8d8d-4caa20ca70a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:59.894515 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aadb0a-d371-450b-8d8d-4caa20ca70a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:59.894795 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aadb0a-d371-450b-8d8d-4caa20ca70a5 admin admin] PATH_INFO: `/v3/projects/8c0c3fa174f3489a9ded73c31f1e7f6d/users/84448aa24c764aa6a598de75fe17e8ec/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:59.901109 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 877/1749] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:08:59 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:08:59.906357 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33fa63a-7d32-40bb-b917-4f3352b5c478 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:59.910837 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aadb0a-d371-450b-8d8d-4caa20ca70a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c0c3fa174f3489a9ded73c31f1e7f6d, user_id=84448aa24c764aa6a598de75fe17e8ec, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:59.913526 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aadb0a-d371-450b-8d8d-4caa20ca70a5 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:59.917185 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33fa63a-7d32-40bb-b917-4f3352b5c478 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:08:59.922854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb8bfa-4372-45dd-815f-ed87c95c8d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:08:59.923014 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb8bfa-4372-45dd-815f-ed87c95c8d02 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:08:59.923236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb8bfa-4372-45dd-815f-ed87c95c8d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:08:59.923549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb8bfa-4372-45dd-815f-ed87c95c8d02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:08:59.924219 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb8bfa-4372-45dd-815f-ed87c95c8d02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:08:59.926171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb8bfa-4372-45dd-815f-ed87c95c8d02 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:08:59.927526 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 873/1750] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:08:59 2022] PUT /identity/v3/projects/8c0c3fa174f3489a9ded73c31f1e7f6d/users/84448aa24c764aa6a598de75fe17e8ec/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:08:59.931618 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aadb0a-d371-450b-8d8d-4caa20ca70a5 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:08:59.948615 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:59.957401 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:08:59.959652 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 878/1751] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:08:59 2022] POST /identity/v3/projects => generated 407 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:08:59.960581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb8bfa-4372-45dd-815f-ed87c95c8d02 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:00.006546 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aadb0a-d371-450b-8d8d-4caa20ca70a5 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:00.023955 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb8bfa-4372-45dd-815f-ed87c95c8d02 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:00.028824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb21ae-8581-4275-942b-331f1ab17379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:00.029453 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb21ae-8581-4275-942b-331f1ab17379 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:00.029734 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb21ae-8581-4275-942b-331f1ab17379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:00.030037 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb21ae-8581-4275-942b-331f1ab17379 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:00.030867 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb21ae-8581-4275-942b-331f1ab17379 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:00.032828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb21ae-8581-4275-942b-331f1ab17379 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:00.055709 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 879/1752] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:08:59 2022] POST /identity/v3/users => generated 310 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:00.059567 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb21ae-8581-4275-942b-331f1ab17379 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:00.066212 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9cd5dd-0b23-40cf-bdab-b1a57f4d12be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:00.066836 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9cd5dd-0b23-40cf-bdab-b1a57f4d12be admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:00.067160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9cd5dd-0b23-40cf-bdab-b1a57f4d12be admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:00.067442 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9cd5dd-0b23-40cf-bdab-b1a57f4d12be admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:00.068195 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9cd5dd-0b23-40cf-bdab-b1a57f4d12be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:00.069807 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb21ae-8581-4275-942b-331f1ab17379 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:00.069936 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9cd5dd-0b23-40cf-bdab-b1a57f4d12be admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:00.074695 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b48aca-0b70-4177-8085-08813a200a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:00.075443 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b48aca-0b70-4177-8085-08813a200a88 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:00.075730 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b48aca-0b70-4177-8085-08813a200a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:00.076008 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b48aca-0b70-4177-8085-08813a200a88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:00.076678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b48aca-0b70-4177-8085-08813a200a88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:00.078925 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b48aca-0b70-4177-8085-08813a200a88 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:00.081295 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 874/1753] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:00.085397 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9cd5dd-0b23-40cf-bdab-b1a57f4d12be admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:00.089412 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 880/1754] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:09:00.093498 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b48aca-0b70-4177-8085-08813a200a88 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:00.096381 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9cd5dd-0b23-40cf-bdab-b1a57f4d12be admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:00.101657 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8c2b0-3f19-413a-bec7-e80d0d33227e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:00.102324 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8c2b0-3f19-413a-bec7-e80d0d33227e admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:00.102612 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8c2b0-3f19-413a-bec7-e80d0d33227e admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:00.103030 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8c2b0-3f19-413a-bec7-e80d0d33227e admin admin] PATH_INFO: `/v3/projects/8c0c3fa174f3489a9ded73c31f1e7f6d/users/84448aa24c764aa6a598de75fe17e8ec/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:00.105146 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8c2b0-3f19-413a-bec7-e80d0d33227e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c0c3fa174f3489a9ded73c31f1e7f6d, user_id=84448aa24c764aa6a598de75fe17e8ec, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:00.107100 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b48aca-0b70-4177-8085-08813a200a88 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:00.107877 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8c2b0-3f19-413a-bec7-e80d0d33227e admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:00.112757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334bcf4-4d8b-4e48-b730-36d44c76617a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:00.113439 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334bcf4-4d8b-4e48-b730-36d44c76617a admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:00.113762 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334bcf4-4d8b-4e48-b730-36d44c76617a admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:00.114060 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334bcf4-4d8b-4e48-b730-36d44c76617a admin admin] PATH_INFO: `/v3/projects/06cff1508c3045268e38dbc204a26654/users/803849595fd947cfa95ee65654f4ffaa/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:00.122612 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 875/1755] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:00 2022] PUT /identity/v3/projects/8c0c3fa174f3489a9ded73c31f1e7f6d/users/84448aa24c764aa6a598de75fe17e8ec/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:09:00.134095 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1402bb-c197-4391-b7cf-a05392efda3a None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:00.134387 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1402bb-c197-4391-b7cf-a05392efda3a None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:00.134668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1402bb-c197-4391-b7cf-a05392efda3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:00.137575 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7334bcf4-4d8b-4e48-b730-36d44c76617a admin admin] RBAC: Authorizing `identity:create_grant(project_id=06cff1508c3045268e38dbc204a26654, user_id=803849595fd947cfa95ee65654f4ffaa, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:00.141277 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7334bcf4-4d8b-4e48-b730-36d44c76617a admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:00.159522 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 881/1756] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:00 2022] PUT /identity/v3/projects/06cff1508c3045268e38dbc204a26654/users/803849595fd947cfa95ee65654f4ffaa/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:09:00.163057 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334bcf4-4d8b-4e48-b730-36d44c76617a admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:00.172509 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-ed1402bb-c197-4391-b7cf-a05392efda3a None None] MFA Rules not processed for user `84448aa24c764aa6a598de75fe17e8ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:00.173986 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ed1402bb-c197-4391-b7cf-a05392efda3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:00.247847 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334bcf4-4d8b-4e48-b730-36d44c76617a admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:00.253849 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4fcd8-0b6d-40e8-b3d0-e9983e0ad4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:00.254066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4fcd8-0b6d-40e8-b3d0-e9983e0ad4bd admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:00.254293 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4fcd8-0b6d-40e8-b3d0-e9983e0ad4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:00.254679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4fcd8-0b6d-40e8-b3d0-e9983e0ad4bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:00.255329 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f4fcd8-0b6d-40e8-b3d0-e9983e0ad4bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:00.257067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f4fcd8-0b6d-40e8-b3d0-e9983e0ad4bd admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:00.261276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 876/1757] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:00 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:09:00.273399 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1402bb-c197-4391-b7cf-a05392efda3a None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:00.275096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 882/1758] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:09:00.277711 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ed1402bb-c197-4391-b7cf-a05392efda3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:00.279927 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4fcd8-0b6d-40e8-b3d0-e9983e0ad4bd admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:00.293352 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4fcd8-0b6d-40e8-b3d0-e9983e0ad4bd admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:00.304897 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33e70c-fd7f-40b7-b03d-849a60d1d67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:00.306073 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33e70c-fd7f-40b7-b03d-849a60d1d67c admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:00.306575 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33e70c-fd7f-40b7-b03d-849a60d1d67c admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:00.307072 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33e70c-fd7f-40b7-b03d-849a60d1d67c admin admin] PATH_INFO: `/v3/projects/06cff1508c3045268e38dbc204a26654/users/803849595fd947cfa95ee65654f4ffaa/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:00.309187 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a33e70c-fd7f-40b7-b03d-849a60d1d67c admin admin] RBAC: Authorizing `identity:create_grant(project_id=06cff1508c3045268e38dbc204a26654, user_id=803849595fd947cfa95ee65654f4ffaa, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:00.312545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a33e70c-fd7f-40b7-b03d-849a60d1d67c admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:00.318751 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:00.323863 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 883/1759] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:00 2022] PUT /identity/v3/projects/06cff1508c3045268e38dbc204a26654/users/803849595fd947cfa95ee65654f4ffaa/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:00.336634 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f6d712-0fc0-416e-b258-e778b86c1474 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:00.336926 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f6d712-0fc0-416e-b258-e778b86c1474 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:00.337317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f6d712-0fc0-416e-b258-e778b86c1474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:00.345276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1402bb-c197-4391-b7cf-a05392efda3a None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:00.365469 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-f2f6d712-0fc0-416e-b258-e778b86c1474 None None] MFA Rules not processed for user `803849595fd947cfa95ee65654f4ffaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:00.366685 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f2f6d712-0fc0-416e-b258-e778b86c1474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:00.394203 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:00.394715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c16dbd-33df-4138-8a5d-f2914bbc0891 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:00.395336 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c16dbd-33df-4138-8a5d-f2914bbc0891 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:00.395627 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c16dbd-33df-4138-8a5d-f2914bbc0891 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:00.395908 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c16dbd-33df-4138-8a5d-f2914bbc0891 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:00.438291 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 884/1760] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:00 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:09:00.449179 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f6d712-0fc0-416e-b258-e778b86c1474 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:00.450532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f2f6d712-0fc0-416e-b258-e778b86c1474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:00.459882 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c16dbd-33df-4138-8a5d-f2914bbc0891 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:00.461361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c16dbd-33df-4138-8a5d-f2914bbc0891 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:00.462758 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f6d712-0fc0-416e-b258-e778b86c1474 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:00.466338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddc2e5-9817-41f6-8188-17bfe85e36f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:00.466972 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddc2e5-9817-41f6-8188-17bfe85e36f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:00.467254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddc2e5-9817-41f6-8188-17bfe85e36f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:00.467528 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddc2e5-9817-41f6-8188-17bfe85e36f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:00.471568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 877/1761] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:00 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:09:00.477205 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ddc2e5-9817-41f6-8188-17bfe85e36f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:00.478616 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ddc2e5-9817-41f6-8188-17bfe85e36f8 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:00.488331 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 885/1762] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:00 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:09:00.563018 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c16dbd-33df-4138-8a5d-f2914bbc0891 service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:00.572717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c16dbd-33df-4138-8a5d-f2914bbc0891 service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:00.576513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9460d-933b-4a3b-9adb-8b31ca78cd4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:00.577835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9460d-933b-4a3b-9adb-8b31ca78cd4b service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:00.578455 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9460d-933b-4a3b-9adb-8b31ca78cd4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:00.578852 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9460d-933b-4a3b-9adb-8b31ca78cd4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:00.649044 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9460d-933b-4a3b-9adb-8b31ca78cd4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:00.650595 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9460d-933b-4a3b-9adb-8b31ca78cd4b service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:00.663421 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 878/1763] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:09:00.776832 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddc2e5-9817-41f6-8188-17bfe85e36f8 service cinder] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:00.785003 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddc2e5-9817-41f6-8188-17bfe85e36f8 service cinder] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:00.788888 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150c947-af9f-40e9-b749-46c1a276e31f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:00.789497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150c947-af9f-40e9-b749-46c1a276e31f service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:00.789779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150c947-af9f-40e9-b749-46c1a276e31f service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:00.790057 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150c947-af9f-40e9-b749-46c1a276e31f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:00.859941 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150c947-af9f-40e9-b749-46c1a276e31f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:00.861758 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150c947-af9f-40e9-b749-46c1a276e31f service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:00.872719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 886/1764] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:00 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:09:10.658070 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=84 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:10.866718 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=51 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:18.381744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9460d-933b-4a3b-9adb-8b31ca78cd4b service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:18.479420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9460d-933b-4a3b-9adb-8b31ca78cd4b service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:18.485116 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd689dbf-dad1-44c7-99f8-2717f84158c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:18.485836 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd689dbf-dad1-44c7-99f8-2717f84158c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:18.486159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd689dbf-dad1-44c7-99f8-2717f84158c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:18.486405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd689dbf-dad1-44c7-99f8-2717f84158c1 admin admin] PATH_INFO: `/v3/users/cd39b19e427145a08929287b48871663` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:18.487560 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd689dbf-dad1-44c7-99f8-2717f84158c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd39b19e427145a08929287b48871663)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:18.489494 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd689dbf-dad1-44c7-99f8-2717f84158c1 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:18.579708 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-bd689dbf-dad1-44c7-99f8-2717f84158c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd39b19e427145a08929287b48871663'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:18.590328 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-bd689dbf-dad1-44c7-99f8-2717f84158c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd39b19e427145a08929287b48871663'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:18.607922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-bd689dbf-dad1-44c7-99f8-2717f84158c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd39b19e427145a08929287b48871663'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:18.609234 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-bd689dbf-dad1-44c7-99f8-2717f84158c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd39b19e427145a08929287b48871663'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:18.610168 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-bd689dbf-dad1-44c7-99f8-2717f84158c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd39b19e427145a08929287b48871663'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:18.636367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 879/1765] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:09:18 2022] DELETE /identity/v3/users/cd39b19e427145a08929287b48871663 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:18.637385 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb5385-c2c3-4665-93c2-15a555032673 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:18.645054 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb5385-c2c3-4665-93c2-15a555032673 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:18.645054 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb5385-c2c3-4665-93c2-15a555032673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:18.668564 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-20eb5385-c2c3-4665-93c2-15a555032673 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:18.669537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-20eb5385-c2c3-4665-93c2-15a555032673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:18.744591 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 887/1766] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:09:18.756503 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd689dbf-dad1-44c7-99f8-2717f84158c1 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:18.758450 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-bd689dbf-dad1-44c7-99f8-2717f84158c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:18.840404 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd689dbf-dad1-44c7-99f8-2717f84158c1 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:18.846059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1cd23-6a8e-4c2b-a2ca-2301b7b0d73f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:18.846784 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1cd23-6a8e-4c2b-a2ca-2301b7b0d73f service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:18.848525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1cd23-6a8e-4c2b-a2ca-2301b7b0d73f service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:18.848525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1cd23-6a8e-4c2b-a2ca-2301b7b0d73f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:18.862760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1cd23-6a8e-4c2b-a2ca-2301b7b0d73f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:18.865832 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1cd23-6a8e-4c2b-a2ca-2301b7b0d73f service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:18.878748 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 880/1767] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:09:18.890762 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb5385-c2c3-4665-93c2-15a555032673 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:18.892224 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-20eb5385-c2c3-4665-93c2-15a555032673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:18.906198 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb5385-c2c3-4665-93c2-15a555032673 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:18.913011 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dba7ea-b58d-42f6-828d-bd6cffb5ffb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:18.913666 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dba7ea-b58d-42f6-828d-bd6cffb5ffb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:18.913965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dba7ea-b58d-42f6-828d-bd6cffb5ffb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:18.914237 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dba7ea-b58d-42f6-828d-bd6cffb5ffb6 admin admin] PATH_INFO: `/v3/users/88501c890a304ad6a96e4d6203fb435e` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:18.915491 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dba7ea-b58d-42f6-828d-bd6cffb5ffb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88501c890a304ad6a96e4d6203fb435e)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:18.917382 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dba7ea-b58d-42f6-828d-bd6cffb5ffb6 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:18.951172 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1cd23-6a8e-4c2b-a2ca-2301b7b0d73f service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:18.953590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-03e1cd23-6a8e-4c2b-a2ca-2301b7b0d73f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:18.970159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1cd23-6a8e-4c2b-a2ca-2301b7b0d73f service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:18.977709 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478844a7-4285-47e8-8316-cc6453e15ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:18.978432 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478844a7-4285-47e8-8316-cc6453e15ee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:18.978717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478844a7-4285-47e8-8316-cc6453e15ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:18.979015 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478844a7-4285-47e8-8316-cc6453e15ee7 admin admin] PATH_INFO: `/v3/users/746acc5102f84af9af6e513e8e902910` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:18.980332 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478844a7-4285-47e8-8316-cc6453e15ee7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=746acc5102f84af9af6e513e8e902910)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:18.982987 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478844a7-4285-47e8-8316-cc6453e15ee7 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:18.989700 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-90dba7ea-b58d-42f6-828d-bd6cffb5ffb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88501c890a304ad6a96e4d6203fb435e'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:18.990613 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-90dba7ea-b58d-42f6-828d-bd6cffb5ffb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88501c890a304ad6a96e4d6203fb435e'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:19.005542 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-90dba7ea-b58d-42f6-828d-bd6cffb5ffb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88501c890a304ad6a96e4d6203fb435e'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:19.006642 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:19.034512 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-90dba7ea-b58d-42f6-828d-bd6cffb5ffb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88501c890a304ad6a96e4d6203fb435e'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:19.046874 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-90dba7ea-b58d-42f6-828d-bd6cffb5ffb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88501c890a304ad6a96e4d6203fb435e'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:19.056175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 888/1768] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:09:18 2022] DELETE /identity/v3/users/88501c890a304ad6a96e4d6203fb435e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:19.062877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64715bbb-243f-4c7c-8438-8be82443e5bd None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:19.063563 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64715bbb-243f-4c7c-8438-8be82443e5bd None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:19.064233 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64715bbb-243f-4c7c-8438-8be82443e5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:19.066717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-478844a7-4285-47e8-8316-cc6453e15ee7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '746acc5102f84af9af6e513e8e902910'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:19.087158 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-478844a7-4285-47e8-8316-cc6453e15ee7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '746acc5102f84af9af6e513e8e902910'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:19.106392 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-64715bbb-243f-4c7c-8438-8be82443e5bd None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:19.108226 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-64715bbb-243f-4c7c-8438-8be82443e5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:19.142268 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-478844a7-4285-47e8-8316-cc6453e15ee7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '746acc5102f84af9af6e513e8e902910'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:19.143282 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-478844a7-4285-47e8-8316-cc6453e15ee7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '746acc5102f84af9af6e513e8e902910'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:19.144037 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-478844a7-4285-47e8-8316-cc6453e15ee7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '746acc5102f84af9af6e513e8e902910'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:19.160685 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 881/1769] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:09:18 2022] DELETE /identity/v3/users/746acc5102f84af9af6e513e8e902910 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:19.164732 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917dcd3-e89d-446f-b4fb-b7dcb35986e1 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:19.165023 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917dcd3-e89d-446f-b4fb-b7dcb35986e1 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:19.165304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917dcd3-e89d-446f-b4fb-b7dcb35986e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:19.189739 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-3917dcd3-e89d-446f-b4fb-b7dcb35986e1 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:19.191131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-3917dcd3-e89d-446f-b4fb-b7dcb35986e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:19.197696 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 889/1770] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:09:19.205887 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64715bbb-243f-4c7c-8438-8be82443e5bd None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:19.207896 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-64715bbb-243f-4c7c-8438-8be82443e5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:19.225739 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:19.269780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 882/1771] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:09:19.280788 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917dcd3-e89d-446f-b4fb-b7dcb35986e1 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:19.284403 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64715bbb-243f-4c7c-8438-8be82443e5bd None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:19.341185 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7093d-054f-41c5-b93c-e20e5e6907db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:19.341812 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7093d-054f-41c5-b93c-e20e5e6907db service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:19.342096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7093d-054f-41c5-b93c-e20e5e6907db service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:19.342374 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7093d-054f-41c5-b93c-e20e5e6907db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:19.353538 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf7093d-054f-41c5-b93c-e20e5e6907db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:19.353717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917dcd3-e89d-446f-b4fb-b7dcb35986e1 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:19.354483 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf7093d-054f-41c5-b93c-e20e5e6907db service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:19.359198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a2c13-82a4-450f-bcac-bd732fc5e410 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:19.360643 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a2c13-82a4-450f-bcac-bd732fc5e410 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:19.361004 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a2c13-82a4-450f-bcac-bd732fc5e410 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:19.361350 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a2c13-82a4-450f-bcac-bd732fc5e410 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:19.365038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 890/1772] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:09:19.373749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a2c13-82a4-450f-bcac-bd732fc5e410 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:19.375389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a2c13-82a4-450f-bcac-bd732fc5e410 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:19.386568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 883/1773] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:09:19.876166 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7093d-054f-41c5-b93c-e20e5e6907db service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:19.878230 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-1bf7093d-054f-41c5-b93c-e20e5e6907db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:19.897666 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7093d-054f-41c5-b93c-e20e5e6907db service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:19.903637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e2e6f-8d1f-4544-b83f-96b835bb07d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:19.904282 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e2e6f-8d1f-4544-b83f-96b835bb07d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:19.904684 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e2e6f-8d1f-4544-b83f-96b835bb07d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:19.905013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e2e6f-8d1f-4544-b83f-96b835bb07d4 admin admin] PATH_INFO: `/v3/projects/3587e476aa664be0bc026f290377e7d7` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:19.907744 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e2e6f-8d1f-4544-b83f-96b835bb07d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3587e476aa664be0bc026f290377e7d7)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:19.909917 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e2e6f-8d1f-4544-b83f-96b835bb07d4 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:19.975355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-bd8e2e6f-8d1f-4544-b83f-96b835bb07d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3587e476aa664be0bc026f290377e7d7', 'request_id': 'req-bd8e2e6f-8d1f-4544-b83f-96b835bb07d4'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:19.985971 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-bd8e2e6f-8d1f-4544-b83f-96b835bb07d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3587e476aa664be0bc026f290377e7d7', 'request_id': 'req-bd8e2e6f-8d1f-4544-b83f-96b835bb07d4'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:19.994096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-bd8e2e6f-8d1f-4544-b83f-96b835bb07d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3587e476aa664be0bc026f290377e7d7', 'request_id': 'req-bd8e2e6f-8d1f-4544-b83f-96b835bb07d4'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:20.010497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-bd8e2e6f-8d1f-4544-b83f-96b835bb07d4 admin admin] The token cache is being invalidate because project 3587e476aa664be0bc026f290377e7d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:09:20.011052 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-bd8e2e6f-8d1f-4544-b83f-96b835bb07d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:20.011712 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-bd8e2e6f-8d1f-4544-b83f-96b835bb07d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:20.017046 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 891/1774] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:09:19 2022] DELETE /identity/v3/projects/3587e476aa664be0bc026f290377e7d7 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:20.018418 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a2c13-82a4-450f-bcac-bd732fc5e410 service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:20.028541 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a2c13-82a4-450f-bcac-bd732fc5e410 service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:20.077764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:20.125084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76225b-662d-470e-91fd-565c4b27c384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:20.125990 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76225b-662d-470e-91fd-565c4b27c384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:20.126678 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76225b-662d-470e-91fd-565c4b27c384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:20.126817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76225b-662d-470e-91fd-565c4b27c384 admin admin] PATH_INFO: `/v3/users/84448aa24c764aa6a598de75fe17e8ec` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:20.128139 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76225b-662d-470e-91fd-565c4b27c384 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84448aa24c764aa6a598de75fe17e8ec)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:20.130349 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76225b-662d-470e-91fd-565c4b27c384 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:20.196371 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4d76225b-662d-470e-91fd-565c4b27c384 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84448aa24c764aa6a598de75fe17e8ec'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:20.205997 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4d76225b-662d-470e-91fd-565c4b27c384 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84448aa24c764aa6a598de75fe17e8ec'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:20.220444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4d76225b-662d-470e-91fd-565c4b27c384 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84448aa24c764aa6a598de75fe17e8ec'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:20.221170 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4d76225b-662d-470e-91fd-565c4b27c384 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84448aa24c764aa6a598de75fe17e8ec'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:20.221780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4d76225b-662d-470e-91fd-565c4b27c384 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84448aa24c764aa6a598de75fe17e8ec'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:20.237766 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 884/1775] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:09:20 2022] DELETE /identity/v3/users/84448aa24c764aa6a598de75fe17e8ec => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:20.282694 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e2e6f-8d1f-4544-b83f-96b835bb07d4 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:20.284306 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-bd8e2e6f-8d1f-4544-b83f-96b835bb07d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:20.300046 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e2e6f-8d1f-4544-b83f-96b835bb07d4 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:20.370923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:20.384063 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0809f8-6195-4559-b96e-aac161dea0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:20.389279 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0809f8-6195-4559-b96e-aac161dea0f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:20.390445 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0809f8-6195-4559-b96e-aac161dea0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:20.391606 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0809f8-6195-4559-b96e-aac161dea0f1 admin admin] PATH_INFO: `/v3/projects/618580c833ef499e9d4bec5c868c2354` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:20.393194 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0809f8-6195-4559-b96e-aac161dea0f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=618580c833ef499e9d4bec5c868c2354)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:20.397678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0809f8-6195-4559-b96e-aac161dea0f1 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:20.458298 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76225b-662d-470e-91fd-565c4b27c384 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:20.459896 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-4d76225b-662d-470e-91fd-565c4b27c384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:20.471139 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-2d0809f8-6195-4559-b96e-aac161dea0f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '618580c833ef499e9d4bec5c868c2354', 'request_id': 'req-2d0809f8-6195-4559-b96e-aac161dea0f1'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:20.485513 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-2d0809f8-6195-4559-b96e-aac161dea0f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '618580c833ef499e9d4bec5c868c2354', 'request_id': 'req-2d0809f8-6195-4559-b96e-aac161dea0f1'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:20.491581 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76225b-662d-470e-91fd-565c4b27c384 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:20.493460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-2d0809f8-6195-4559-b96e-aac161dea0f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '618580c833ef499e9d4bec5c868c2354', 'request_id': 'req-2d0809f8-6195-4559-b96e-aac161dea0f1'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:20.523716 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-2d0809f8-6195-4559-b96e-aac161dea0f1 admin admin] The token cache is being invalidate because project 618580c833ef499e9d4bec5c868c2354 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:09:20.524565 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-2d0809f8-6195-4559-b96e-aac161dea0f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:20.526607 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-2d0809f8-6195-4559-b96e-aac161dea0f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:20.529097 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 892/1776] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:09:20 2022] DELETE /identity/v3/projects/618580c833ef499e9d4bec5c868c2354 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:20.534336 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0809f8-6195-4559-b96e-aac161dea0f1 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:20.537092 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2d0809f8-6195-4559-b96e-aac161dea0f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:20.595486 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96b4c6-2efd-4f6b-8de5-203aadbe069e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:20.597035 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96b4c6-2efd-4f6b-8de5-203aadbe069e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:20.597630 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96b4c6-2efd-4f6b-8de5-203aadbe069e admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:20.598140 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96b4c6-2efd-4f6b-8de5-203aadbe069e admin admin] PATH_INFO: `/v3/projects/268a58134e11444a8f34c7f1ffd304d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:20.600978 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c96b4c6-2efd-4f6b-8de5-203aadbe069e admin admin] RBAC: Authorizing `identity:delete_project(project_id=268a58134e11444a8f34c7f1ffd304d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:20.603488 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c96b4c6-2efd-4f6b-8de5-203aadbe069e admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:20.654997 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0809f8-6195-4559-b96e-aac161dea0f1 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:20.664374 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517292d-19db-4b0a-94ce-a11ba54cd52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:20.665500 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517292d-19db-4b0a-94ce-a11ba54cd52a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:20.665974 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517292d-19db-4b0a-94ce-a11ba54cd52a admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:20.666460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517292d-19db-4b0a-94ce-a11ba54cd52a admin admin] PATH_INFO: `/v3/users/803849595fd947cfa95ee65654f4ffaa` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:20.668415 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517292d-19db-4b0a-94ce-a11ba54cd52a admin admin] RBAC: Authorizing `identity:delete_user(user_id=803849595fd947cfa95ee65654f4ffaa)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:20.674777 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517292d-19db-4b0a-94ce-a11ba54cd52a admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:20.722647 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-5c96b4c6-2efd-4f6b-8de5-203aadbe069e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '268a58134e11444a8f34c7f1ffd304d6', 'request_id': 'req-5c96b4c6-2efd-4f6b-8de5-203aadbe069e'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:20.738920 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-5c96b4c6-2efd-4f6b-8de5-203aadbe069e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '268a58134e11444a8f34c7f1ffd304d6', 'request_id': 'req-5c96b4c6-2efd-4f6b-8de5-203aadbe069e'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:20.747852 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:20.758556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-5c96b4c6-2efd-4f6b-8de5-203aadbe069e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '268a58134e11444a8f34c7f1ffd304d6', 'request_id': 'req-5c96b4c6-2efd-4f6b-8de5-203aadbe069e'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:20.785030 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-5c96b4c6-2efd-4f6b-8de5-203aadbe069e admin admin] The token cache is being invalidate because project 268a58134e11444a8f34c7f1ffd304d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:09:20.785688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-5c96b4c6-2efd-4f6b-8de5-203aadbe069e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:20.786844 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-5c96b4c6-2efd-4f6b-8de5-203aadbe069e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:20.789310 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 885/1777] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:09:20 2022] DELETE /identity/v3/projects/268a58134e11444a8f34c7f1ffd304d6 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:20.793496 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96b4c6-2efd-4f6b-8de5-203aadbe069e admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:20.796610 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5c96b4c6-2efd-4f6b-8de5-203aadbe069e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:20.803353 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-e517292d-19db-4b0a-94ce-a11ba54cd52a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '803849595fd947cfa95ee65654f4ffaa'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:20.804400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-e517292d-19db-4b0a-94ce-a11ba54cd52a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '803849595fd947cfa95ee65654f4ffaa'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:20.819012 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-e517292d-19db-4b0a-94ce-a11ba54cd52a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '803849595fd947cfa95ee65654f4ffaa'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:20.843266 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-e517292d-19db-4b0a-94ce-a11ba54cd52a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '803849595fd947cfa95ee65654f4ffaa'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:20.860401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-e517292d-19db-4b0a-94ce-a11ba54cd52a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '803849595fd947cfa95ee65654f4ffaa'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:20.876432 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 893/1778] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:09:20 2022] DELETE /identity/v3/users/803849595fd947cfa95ee65654f4ffaa => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:20.951671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517292d-19db-4b0a-94ce-a11ba54cd52a admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:20.954836 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-e517292d-19db-4b0a-94ce-a11ba54cd52a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:20.967032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96b4c6-2efd-4f6b-8de5-203aadbe069e admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:20.970960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:21.113110 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8dc6b-45bc-4f30-bf91-15f679769c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:21.113757 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8dc6b-45bc-4f30-bf91-15f679769c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:21.114038 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8dc6b-45bc-4f30-bf91-15f679769c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:21.114314 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8dc6b-45bc-4f30-bf91-15f679769c4b admin admin] PATH_INFO: `/v3/users/6b689f64354f4ed99d56172a05b23725` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:21.115769 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd8dc6b-45bc-4f30-bf91-15f679769c4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b689f64354f4ed99d56172a05b23725)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:21.118370 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd8dc6b-45bc-4f30-bf91-15f679769c4b admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:21.170061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517292d-19db-4b0a-94ce-a11ba54cd52a admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:21.176445 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c582c-9513-48a4-bf66-09ada550a0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:21.177702 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c582c-9513-48a4-bf66-09ada550a0e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:21.177702 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c582c-9513-48a4-bf66-09ada550a0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:21.177880 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c582c-9513-48a4-bf66-09ada550a0e7 admin admin] PATH_INFO: `/v3/users/234acc4aa5e943118f737708fe9352f1` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:21.179498 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c582c-9513-48a4-bf66-09ada550a0e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=234acc4aa5e943118f737708fe9352f1)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:21.181450 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c582c-9513-48a4-bf66-09ada550a0e7 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:21.184241 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-cdd8dc6b-45bc-4f30-bf91-15f679769c4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b689f64354f4ed99d56172a05b23725'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:21.192786 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-cdd8dc6b-45bc-4f30-bf91-15f679769c4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b689f64354f4ed99d56172a05b23725'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:21.207956 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-cdd8dc6b-45bc-4f30-bf91-15f679769c4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b689f64354f4ed99d56172a05b23725'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:21.208746 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-cdd8dc6b-45bc-4f30-bf91-15f679769c4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b689f64354f4ed99d56172a05b23725'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:21.209394 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-cdd8dc6b-45bc-4f30-bf91-15f679769c4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b689f64354f4ed99d56172a05b23725'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:21.226510 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 886/1779] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:09:20 2022] DELETE /identity/v3/users/6b689f64354f4ed99d56172a05b23725 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:21.227746 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8dc6b-45bc-4f30-bf91-15f679769c4b admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:21.229533 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-cdd8dc6b-45bc-4f30-bf91-15f679769c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:21.247306 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:21.262106 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-ac6c582c-9513-48a4-bf66-09ada550a0e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '234acc4aa5e943118f737708fe9352f1'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:21.262947 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-ac6c582c-9513-48a4-bf66-09ada550a0e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '234acc4aa5e943118f737708fe9352f1'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:21.276198 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-ac6c582c-9513-48a4-bf66-09ada550a0e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '234acc4aa5e943118f737708fe9352f1'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:21.296113 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-ac6c582c-9513-48a4-bf66-09ada550a0e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '234acc4aa5e943118f737708fe9352f1'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:21.309032 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-ac6c582c-9513-48a4-bf66-09ada550a0e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '234acc4aa5e943118f737708fe9352f1'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:21.316914 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8dc6b-45bc-4f30-bf91-15f679769c4b admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:21.320598 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 894/1780] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:09:20 2022] DELETE /identity/v3/users/234acc4aa5e943118f737708fe9352f1 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:09:21.325776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5fbe9-5a44-4462-a41c-76059ea7fb48 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:21.326275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5fbe9-5a44-4462-a41c-76059ea7fb48 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:21.326759 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5fbe9-5a44-4462-a41c-76059ea7fb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:21.329050 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80521182-1227-408c-9fde-230b17ffd738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:21.329899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80521182-1227-408c-9fde-230b17ffd738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:21.330343 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80521182-1227-408c-9fde-230b17ffd738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:21.330724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80521182-1227-408c-9fde-230b17ffd738 admin admin] PATH_INFO: `/v3/projects/8c0c3fa174f3489a9ded73c31f1e7f6d` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:21.332064 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80521182-1227-408c-9fde-230b17ffd738 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c0c3fa174f3489a9ded73c31f1e7f6d)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:21.335082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80521182-1227-408c-9fde-230b17ffd738 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:21.354890 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-b7d5fbe9-5a44-4462-a41c-76059ea7fb48 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:21.356252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b7d5fbe9-5a44-4462-a41c-76059ea7fb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:21.432365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-80521182-1227-408c-9fde-230b17ffd738 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c0c3fa174f3489a9ded73c31f1e7f6d', 'request_id': 'req-80521182-1227-408c-9fde-230b17ffd738'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:21.455961 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-80521182-1227-408c-9fde-230b17ffd738 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c0c3fa174f3489a9ded73c31f1e7f6d', 'request_id': 'req-80521182-1227-408c-9fde-230b17ffd738'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:21.466444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-80521182-1227-408c-9fde-230b17ffd738 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c0c3fa174f3489a9ded73c31f1e7f6d', 'request_id': 'req-80521182-1227-408c-9fde-230b17ffd738'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:21.482993 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 895/1781] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:21.485395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-80521182-1227-408c-9fde-230b17ffd738 admin admin] The token cache is being invalidate because project 8c0c3fa174f3489a9ded73c31f1e7f6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:09:21.485781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-80521182-1227-408c-9fde-230b17ffd738 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:21.486532 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-80521182-1227-408c-9fde-230b17ffd738 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:21.488544 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 887/1782] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:09:21 2022] DELETE /identity/v3/projects/8c0c3fa174f3489a9ded73c31f1e7f6d => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:21.490393 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5fbe9-5a44-4462-a41c-76059ea7fb48 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:21.492116 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b7d5fbe9-5a44-4462-a41c-76059ea7fb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:21.494454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79325de4-cdb0-4a95-b503-4907bd76c1b5 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:21.494961 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79325de4-cdb0-4a95-b503-4907bd76c1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:21.495443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79325de4-cdb0-4a95-b503-4907bd76c1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:21.532172 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:21.549734 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-79325de4-cdb0-4a95-b503-4907bd76c1b5 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:21.551742 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-79325de4-cdb0-4a95-b503-4907bd76c1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:21.571226 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5fbe9-5a44-4462-a41c-76059ea7fb48 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:21.659373 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d9477-99ad-457c-830f-9f7a8f3aad8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:21.659588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d9477-99ad-457c-830f-9f7a8f3aad8e service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:21.659833 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d9477-99ad-457c-830f-9f7a8f3aad8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:21.660381 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d9477-99ad-457c-830f-9f7a8f3aad8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:21.662617 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f89d9477-99ad-457c-830f-9f7a8f3aad8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:21.663808 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 888/1783] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:09:21.664859 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79325de4-cdb0-4a95-b503-4907bd76c1b5 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:21.666525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-79325de4-cdb0-4a95-b503-4907bd76c1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:21.677332 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d9477-99ad-457c-830f-9f7a8f3aad8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:21.679096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d9477-99ad-457c-830f-9f7a8f3aad8e service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:21.685376 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79325de4-cdb0-4a95-b503-4907bd76c1b5 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:21.692042 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c23541-d130-4ea7-88d8-fab740ce4e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:21.693368 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c23541-d130-4ea7-88d8-fab740ce4e66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:21.693979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c23541-d130-4ea7-88d8-fab740ce4e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:21.694193 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 896/1784] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:09:21.694922 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c23541-d130-4ea7-88d8-fab740ce4e66 admin admin] PATH_INFO: `/v3/projects/7d9c4e0a9bb247feb49f9c7ef6393ca4` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:21.695407 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e36ca-5eac-4e1f-8565-923cc52adf92 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:21.695698 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e36ca-5eac-4e1f-8565-923cc52adf92 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:21.696014 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e36ca-5eac-4e1f-8565-923cc52adf92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:21.697895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c23541-d130-4ea7-88d8-fab740ce4e66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d9c4e0a9bb247feb49f9c7ef6393ca4)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:21.701207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c23541-d130-4ea7-88d8-fab740ce4e66 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:21.732133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-779e36ca-5eac-4e1f-8565-923cc52adf92 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:21.736257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-779e36ca-5eac-4e1f-8565-923cc52adf92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:21.757153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 897/1785] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:21.757153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e36ca-5eac-4e1f-8565-923cc52adf92 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:21.757970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-779e36ca-5eac-4e1f-8565-923cc52adf92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:21.781148 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:21.787014 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e36ca-5eac-4e1f-8565-923cc52adf92 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:21.794756 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f4c23541-d130-4ea7-88d8-fab740ce4e66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d9c4e0a9bb247feb49f9c7ef6393ca4', 'request_id': 'req-f4c23541-d130-4ea7-88d8-fab740ce4e66'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:21.808748 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f4c23541-d130-4ea7-88d8-fab740ce4e66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d9c4e0a9bb247feb49f9c7ef6393ca4', 'request_id': 'req-f4c23541-d130-4ea7-88d8-fab740ce4e66'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:21.826255 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f4c23541-d130-4ea7-88d8-fab740ce4e66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d9c4e0a9bb247feb49f9c7ef6393ca4', 'request_id': 'req-f4c23541-d130-4ea7-88d8-fab740ce4e66'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:21.846311 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f4c23541-d130-4ea7-88d8-fab740ce4e66 admin admin] The token cache is being invalidate because project 7d9c4e0a9bb247feb49f9c7ef6393ca4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:09:21.846762 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f4c23541-d130-4ea7-88d8-fab740ce4e66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:21.847537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f4c23541-d130-4ea7-88d8-fab740ce4e66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:21.855866 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 889/1786] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:09:21 2022] DELETE /identity/v3/projects/7d9c4e0a9bb247feb49f9c7ef6393ca4 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:21.855866 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c23541-d130-4ea7-88d8-fab740ce4e66 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:21.858346 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f4c23541-d130-4ea7-88d8-fab740ce4e66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:21.906628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf7b37-2af3-436b-a5ad-1e2525559be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:21.908141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf7b37-2af3-436b-a5ad-1e2525559be8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:21.908847 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf7b37-2af3-436b-a5ad-1e2525559be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:21.909300 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf7b37-2af3-436b-a5ad-1e2525559be8 admin admin] PATH_INFO: `/v3/projects/06cff1508c3045268e38dbc204a26654` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:21.911465 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf7b37-2af3-436b-a5ad-1e2525559be8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06cff1508c3045268e38dbc204a26654)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:21.913998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf7b37-2af3-436b-a5ad-1e2525559be8 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:22.031275 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c23541-d130-4ea7-88d8-fab740ce4e66 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:22.032148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-92bf7b37-2af3-436b-a5ad-1e2525559be8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06cff1508c3045268e38dbc204a26654', 'request_id': 'req-92bf7b37-2af3-436b-a5ad-1e2525559be8'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:22.059976 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-92bf7b37-2af3-436b-a5ad-1e2525559be8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06cff1508c3045268e38dbc204a26654', 'request_id': 'req-92bf7b37-2af3-436b-a5ad-1e2525559be8'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:22.078933 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-92bf7b37-2af3-436b-a5ad-1e2525559be8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06cff1508c3045268e38dbc204a26654', 'request_id': 'req-92bf7b37-2af3-436b-a5ad-1e2525559be8'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:22.103880 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-92bf7b37-2af3-436b-a5ad-1e2525559be8 admin admin] The token cache is being invalidate because project 06cff1508c3045268e38dbc204a26654 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:09:22.104968 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-92bf7b37-2af3-436b-a5ad-1e2525559be8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:22.105785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-92bf7b37-2af3-436b-a5ad-1e2525559be8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:22.113063 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 898/1787] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:09:21 2022] DELETE /identity/v3/projects/06cff1508c3045268e38dbc204a26654 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:22.117080 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bcde3-26aa-4193-b87f-3df762d8466a None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:22.117370 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bcde3-26aa-4193-b87f-3df762d8466a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:22.117654 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bcde3-26aa-4193-b87f-3df762d8466a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:22.127310 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e498fb-b896-47da-a810-17a58a225ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:22.127929 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e498fb-b896-47da-a810-17a58a225ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:22.128222 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e498fb-b896-47da-a810-17a58a225ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:22.128534 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e498fb-b896-47da-a810-17a58a225ae1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:22.130019 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e498fb-b896-47da-a810-17a58a225ae1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:09:22.130019 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e498fb-b896-47da-a810-17a58a225ae1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:22.131578 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e498fb-b896-47da-a810-17a58a225ae1 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:22.137204 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:22.144751 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-bf0bcde3-26aa-4193-b87f-3df762d8466a None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:22.146604 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-bf0bcde3-26aa-4193-b87f-3df762d8466a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:22.147258 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 890/1788] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:09:22.148936 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd5e27-be9b-4e1e-a6b1-1a64bdf97bac None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:22.149120 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd5e27-be9b-4e1e-a6b1-1a64bdf97bac None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:22.149401 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd5e27-be9b-4e1e-a6b1-1a64bdf97bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:22.165600 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 899/1789] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:22.165600 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bcde3-26aa-4193-b87f-3df762d8466a None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:22.169669 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-bf0bcde3-26aa-4193-b87f-3df762d8466a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:22.175272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-89fd5e27-be9b-4e1e-a6b1-1a64bdf97bac None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:22.176512 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-89fd5e27-be9b-4e1e-a6b1-1a64bdf97bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:22.184102 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bcde3-26aa-4193-b87f-3df762d8466a None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:22.193565 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa9468-ef39-4b78-adb2-ee04e5508766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:22.194412 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa9468-ef39-4b78-adb2-ee04e5508766 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:22.194964 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa9468-ef39-4b78-adb2-ee04e5508766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:22.195416 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa9468-ef39-4b78-adb2-ee04e5508766 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:22.196184 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 891/1790] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:22.196468 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa9468-ef39-4b78-adb2-ee04e5508766 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:22.197620 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2741a-fcca-43b3-bae6-a23b502341aa None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:22.197924 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2741a-fcca-43b3-bae6-a23b502341aa None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:22.198207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2741a-fcca-43b3-bae6-a23b502341aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:22.200161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa9468-ef39-4b78-adb2-ee04e5508766 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:22.225041 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-38d2741a-fcca-43b3-bae6-a23b502341aa None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:22.226839 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-38d2741a-fcca-43b3-bae6-a23b502341aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:22.242976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 892/1791] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:22.243184 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 900/1792] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:09:22 2022] POST /identity/v3/projects => generated 377 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:22.243289 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03c9cc-a6ef-4dac-b3dc-b22330daffdc None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:22.243388 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03c9cc-a6ef-4dac-b3dc-b22330daffdc None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:22.243716 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03c9cc-a6ef-4dac-b3dc-b22330daffdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:22.244534 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa9468-ef39-4b78-adb2-ee04e5508766 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:22.281724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-cb03c9cc-a6ef-4dac-b3dc-b22330daffdc None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:22.283491 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-cb03c9cc-a6ef-4dac-b3dc-b22330daffdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:22.340885 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa9468-ef39-4b78-adb2-ee04e5508766 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:22.347665 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:22.347824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6b9a8-7000-4091-81c2-8560deb208d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:22.348770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6b9a8-7000-4091-81c2-8560deb208d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:22.349346 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6b9a8-7000-4091-81c2-8560deb208d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:22.349693 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6b9a8-7000-4091-81c2-8560deb208d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:22.351063 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6b9a8-7000-4091-81c2-8560deb208d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:09:22.352090 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6b9a8-7000-4091-81c2-8560deb208d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:22.354431 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6b9a8-7000-4091-81c2-8560deb208d9 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:22.371275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 901/1793] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:09:22.371728 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 893/1794] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:22.372908 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03c9cc-a6ef-4dac-b3dc-b22330daffdc None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:22.373290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6b9a8-7000-4091-81c2-8560deb208d9 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:22.375296 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-cb03c9cc-a6ef-4dac-b3dc-b22330daffdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:22.397939 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6b9a8-7000-4091-81c2-8560deb208d9 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:22.407432 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13231d6-8a8c-44ec-b48f-3ccdeeac17af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:22.408175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13231d6-8a8c-44ec-b48f-3ccdeeac17af admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:22.408614 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13231d6-8a8c-44ec-b48f-3ccdeeac17af admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:22.409223 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13231d6-8a8c-44ec-b48f-3ccdeeac17af admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:22.409982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13231d6-8a8c-44ec-b48f-3ccdeeac17af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:22.412918 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13231d6-8a8c-44ec-b48f-3ccdeeac17af admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:22.413359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03c9cc-a6ef-4dac-b3dc-b22330daffdc None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:22.418979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3a026-0808-49af-8d9b-cba479bf23c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:22.419636 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3a026-0808-49af-8d9b-cba479bf23c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:22.419920 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3a026-0808-49af-8d9b-cba479bf23c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:22.420202 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3a026-0808-49af-8d9b-cba479bf23c0 admin admin] PATH_INFO: `/v3/projects/d36d19cfc6c8483ebd1d80daf514d361` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:22.421941 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3a026-0808-49af-8d9b-cba479bf23c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d36d19cfc6c8483ebd1d80daf514d361)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:22.424299 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3a026-0808-49af-8d9b-cba479bf23c0 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:22.442393 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 902/1795] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:09:22 2022] POST /identity/v3/users => generated 289 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:22.443701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13231d6-8a8c-44ec-b48f-3ccdeeac17af admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:22.455842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13231d6-8a8c-44ec-b48f-3ccdeeac17af admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:22.464983 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a97f3df-0437-4bbc-b3c2-1a9cad0df27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:22.465644 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97f3df-0437-4bbc-b3c2-1a9cad0df27d admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:22.465928 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97f3df-0437-4bbc-b3c2-1a9cad0df27d admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:22.467179 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97f3df-0437-4bbc-b3c2-1a9cad0df27d admin admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:22.467909 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a97f3df-0437-4bbc-b3c2-1a9cad0df27d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:09:22.468214 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a97f3df-0437-4bbc-b3c2-1a9cad0df27d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:22.470132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a97f3df-0437-4bbc-b3c2-1a9cad0df27d admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:22.498143 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 903/1796] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:09:22.499770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a97f3df-0437-4bbc-b3c2-1a9cad0df27d admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:22.503729 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-abc3a026-0808-49af-8d9b-cba479bf23c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd36d19cfc6c8483ebd1d80daf514d361', 'request_id': 'req-abc3a026-0808-49af-8d9b-cba479bf23c0'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:22.512219 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a97f3df-0437-4bbc-b3c2-1a9cad0df27d admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:22.512344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-abc3a026-0808-49af-8d9b-cba479bf23c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd36d19cfc6c8483ebd1d80daf514d361', 'request_id': 'req-abc3a026-0808-49af-8d9b-cba479bf23c0'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:22.523647 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-abc3a026-0808-49af-8d9b-cba479bf23c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd36d19cfc6c8483ebd1d80daf514d361', 'request_id': 'req-abc3a026-0808-49af-8d9b-cba479bf23c0'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:22.542942 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-abc3a026-0808-49af-8d9b-cba479bf23c0 admin admin] The token cache is being invalidate because project d36d19cfc6c8483ebd1d80daf514d361 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:09:22.543244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-abc3a026-0808-49af-8d9b-cba479bf23c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:22.544036 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-abc3a026-0808-49af-8d9b-cba479bf23c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:22.548971 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 894/1797] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:09:22 2022] DELETE /identity/v3/projects/d36d19cfc6c8483ebd1d80daf514d361 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:22.548971 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3a026-0808-49af-8d9b-cba479bf23c0 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:22.550364 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-abc3a026-0808-49af-8d9b-cba479bf23c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:22.606715 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58edb3cf-0ccc-4853-9c75-0f9f7d2e21d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:22.607914 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edb3cf-0ccc-4853-9c75-0f9f7d2e21d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:22.608329 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edb3cf-0ccc-4853-9c75-0f9f7d2e21d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:22.609307 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edb3cf-0ccc-4853-9c75-0f9f7d2e21d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:22.610139 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58edb3cf-0ccc-4853-9c75-0f9f7d2e21d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:22.613818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58edb3cf-0ccc-4853-9c75-0f9f7d2e21d7 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:22.633062 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3a026-0808-49af-8d9b-cba479bf23c0 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:22.639242 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2245ff3c-0fe2-4dfa-8109-d86879893aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:22.640711 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245ff3c-0fe2-4dfa-8109-d86879893aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:22.641252 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245ff3c-0fe2-4dfa-8109-d86879893aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:22.641751 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245ff3c-0fe2-4dfa-8109-d86879893aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:22.642378 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2245ff3c-0fe2-4dfa-8109-d86879893aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:22.644009 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2245ff3c-0fe2-4dfa-8109-d86879893aa1 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:22.654914 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 895/1798] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:22.655966 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2245ff3c-0fe2-4dfa-8109-d86879893aa1 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:22.658965 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-2245ff3c-0fe2-4dfa-8109-d86879893aa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:22.663061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 904/1799] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:09:22 2022] POST /identity/v3/projects => generated 391 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:22.663719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58edb3cf-0ccc-4853-9c75-0f9f7d2e21d7 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:22.665396 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-58edb3cf-0ccc-4853-9c75-0f9f7d2e21d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:22.675678 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2245ff3c-0fe2-4dfa-8109-d86879893aa1 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:22.709202 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:22.741235 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c29c09-edbc-4e1f-a5ea-39eb86d02dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:22.741867 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c29c09-edbc-4e1f-a5ea-39eb86d02dcc admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:22.742160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c29c09-edbc-4e1f-a5ea-39eb86d02dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:22.742427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c29c09-edbc-4e1f-a5ea-39eb86d02dcc admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:22.743136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c29c09-edbc-4e1f-a5ea-39eb86d02dcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:22.745350 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c29c09-edbc-4e1f-a5ea-39eb86d02dcc admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:22.773051 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:22.778128 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58edb3cf-0ccc-4853-9c75-0f9f7d2e21d7 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:22.785256 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 896/1800] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:09:22 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:22.785256 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c29c09-edbc-4e1f-a5ea-39eb86d02dcc admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:22.859652 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9685e85f-cff0-4c92-a377-5d13f3382db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:22.860406 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9685e85f-cff0-4c92-a377-5d13f3382db7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:22.860799 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9685e85f-cff0-4c92-a377-5d13f3382db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:22.861337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9685e85f-cff0-4c92-a377-5d13f3382db7 admin admin] PATH_INFO: `/v3/users/d784702786964cfe84c28b590fc701d7` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:22.862552 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9685e85f-cff0-4c92-a377-5d13f3382db7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d784702786964cfe84c28b590fc701d7)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:22.864507 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c29c09-edbc-4e1f-a5ea-39eb86d02dcc admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:22.865121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9685e85f-cff0-4c92-a377-5d13f3382db7 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:22.870699 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55feacf0-4ba8-4c34-aeaf-45ce9eae23c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:22.871551 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55feacf0-4ba8-4c34-aeaf-45ce9eae23c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:22.871994 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55feacf0-4ba8-4c34-aeaf-45ce9eae23c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:22.872275 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55feacf0-4ba8-4c34-aeaf-45ce9eae23c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:22.872967 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55feacf0-4ba8-4c34-aeaf-45ce9eae23c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:22.875089 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55feacf0-4ba8-4c34-aeaf-45ce9eae23c1 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:22.886728 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 897/1801] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:22.887808 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55feacf0-4ba8-4c34-aeaf-45ce9eae23c1 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:22.890856 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-55feacf0-4ba8-4c34-aeaf-45ce9eae23c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:22.904845 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55feacf0-4ba8-4c34-aeaf-45ce9eae23c1 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:22.910570 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676ab6d-4fc4-4897-9675-5994ae546933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:22.911270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676ab6d-4fc4-4897-9675-5994ae546933 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:22.911663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676ab6d-4fc4-4897-9675-5994ae546933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:22.912019 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676ab6d-4fc4-4897-9675-5994ae546933 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:22.913449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676ab6d-4fc4-4897-9675-5994ae546933 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:22.914771 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676ab6d-4fc4-4897-9675-5994ae546933 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:22.920559 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-9685e85f-cff0-4c92-a377-5d13f3382db7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd784702786964cfe84c28b590fc701d7'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:22.921570 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-9685e85f-cff0-4c92-a377-5d13f3382db7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd784702786964cfe84c28b590fc701d7'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:22.929478 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-9685e85f-cff0-4c92-a377-5d13f3382db7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd784702786964cfe84c28b590fc701d7'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:22.942406 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 898/1802] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:09:22 2022] POST /identity/v3/users => generated 296 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:22.942406 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676ab6d-4fc4-4897-9675-5994ae546933 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:22.947049 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-9685e85f-cff0-4c92-a377-5d13f3382db7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd784702786964cfe84c28b590fc701d7'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:22.951914 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676ab6d-4fc4-4897-9675-5994ae546933 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:22.959890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522bf83-eac5-42e3-8a6a-23b0e7c6dbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:22.960552 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522bf83-eac5-42e3-8a6a-23b0e7c6dbc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:22.960845 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522bf83-eac5-42e3-8a6a-23b0e7c6dbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:22.961122 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522bf83-eac5-42e3-8a6a-23b0e7c6dbc2 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:22.961296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-9685e85f-cff0-4c92-a377-5d13f3382db7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd784702786964cfe84c28b590fc701d7'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:22.961796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522bf83-eac5-42e3-8a6a-23b0e7c6dbc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:22.964062 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522bf83-eac5-42e3-8a6a-23b0e7c6dbc2 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:22.968427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 905/1803] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:09:22 2022] DELETE /identity/v3/users/d784702786964cfe84c28b590fc701d7 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:22.972350 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9685e85f-cff0-4c92-a377-5d13f3382db7 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:22.974627 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9685e85f-cff0-4c92-a377-5d13f3382db7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:22.994883 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 899/1804] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:09:22 2022] POST /identity/v3/users => generated 294 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:22.995932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522bf83-eac5-42e3-8a6a-23b0e7c6dbc2 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:22.998022 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1522bf83-eac5-42e3-8a6a-23b0e7c6dbc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:23.014583 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522bf83-eac5-42e3-8a6a-23b0e7c6dbc2 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:23.053403 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:23.076965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9685e85f-cff0-4c92-a377-5d13f3382db7 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:23.084862 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff67865-9abc-4b0e-a4d7-17a78300b038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:23.085946 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff67865-9abc-4b0e-a4d7-17a78300b038 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:23.086341 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff67865-9abc-4b0e-a4d7-17a78300b038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:23.086731 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff67865-9abc-4b0e-a4d7-17a78300b038 admin admin] PATH_INFO: `/v3/projects/5175d054cb01475ab24f91b354ab7011/users/7e0cd8a9bb5442cc84130943983d0f36/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:23.107089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff67865-9abc-4b0e-a4d7-17a78300b038 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5175d054cb01475ab24f91b354ab7011, user_id=7e0cd8a9bb5442cc84130943983d0f36, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:23.111000 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff67865-9abc-4b0e-a4d7-17a78300b038 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:23.133417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 906/1805] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:22 2022] PUT /identity/v3/projects/5175d054cb01475ab24f91b354ab7011/users/7e0cd8a9bb5442cc84130943983d0f36/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:23.134740 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff67865-9abc-4b0e-a4d7-17a78300b038 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:23.137048 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-5ff67865-9abc-4b0e-a4d7-17a78300b038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:23.144453 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fc3dd-7d56-4afe-a7cd-6e64228599d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:23.146427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fc3dd-7d56-4afe-a7cd-6e64228599d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:23.146723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fc3dd-7d56-4afe-a7cd-6e64228599d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:23.147002 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fc3dd-7d56-4afe-a7cd-6e64228599d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:23.147659 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129fc3dd-7d56-4afe-a7cd-6e64228599d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:23.149493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129fc3dd-7d56-4afe-a7cd-6e64228599d8 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:23.165842 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 900/1806] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:23.176579 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cbba75-c51c-45ea-b2ae-1db4940dc1a9 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:23.176579 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cbba75-c51c-45ea-b2ae-1db4940dc1a9 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:23.176579 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cbba75-c51c-45ea-b2ae-1db4940dc1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:23.222968 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-78cbba75-c51c-45ea-b2ae-1db4940dc1a9 None None] MFA Rules not processed for user `7e0cd8a9bb5442cc84130943983d0f36`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:23.224826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-78cbba75-c51c-45ea-b2ae-1db4940dc1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:23.228258 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff67865-9abc-4b0e-a4d7-17a78300b038 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:23.235145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c60a6-e6ca-4347-8d54-584d6218e48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:23.236115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c60a6-e6ca-4347-8d54-584d6218e48c admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:23.236520 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c60a6-e6ca-4347-8d54-584d6218e48c admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:23.236913 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c60a6-e6ca-4347-8d54-584d6218e48c admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:23.237670 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9c60a6-e6ca-4347-8d54-584d6218e48c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:23.239565 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9c60a6-e6ca-4347-8d54-584d6218e48c admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:23.258741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 907/1807] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:23.259272 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c60a6-e6ca-4347-8d54-584d6218e48c admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:23.285119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c60a6-e6ca-4347-8d54-584d6218e48c admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:23.294634 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:23.296570 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f27e5b1-7aa8-4313-ad18-334dda9e6e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:23.297404 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27e5b1-7aa8-4313-ad18-334dda9e6e37 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:23.299257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27e5b1-7aa8-4313-ad18-334dda9e6e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:23.299915 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27e5b1-7aa8-4313-ad18-334dda9e6e37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:23.302337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f27e5b1-7aa8-4313-ad18-334dda9e6e37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:23.304346 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f27e5b1-7aa8-4313-ad18-334dda9e6e37 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:23.326292 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 908/1808] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:23.327727 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f27e5b1-7aa8-4313-ad18-334dda9e6e37 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:23.336060 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 901/1809] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:23.339586 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cbba75-c51c-45ea-b2ae-1db4940dc1a9 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:23.339674 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f27e5b1-7aa8-4313-ad18-334dda9e6e37 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:23.346175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0ad01-a48b-448f-b32c-768e0144e690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:23.347229 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0ad01-a48b-448f-b32c-768e0144e690 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:23.347516 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0ad01-a48b-448f-b32c-768e0144e690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:23.347796 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0ad01-a48b-448f-b32c-768e0144e690 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:23.348448 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae0ad01-a48b-448f-b32c-768e0144e690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:23.349820 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cbba75-c51c-45ea-b2ae-1db4940dc1a9 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:23.350072 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae0ad01-a48b-448f-b32c-768e0144e690 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:23.355456 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd7affb-6a58-4297-ae91-a8307cf873d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:23.356207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7affb-6a58-4297-ae91-a8307cf873d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:23.356714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7affb-6a58-4297-ae91-a8307cf873d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:23.357064 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7affb-6a58-4297-ae91-a8307cf873d1 admin admin] PATH_INFO: `/v3/projects/22784ba66b5247b880f3bf88ae2a900e/users/08480a18e2f9491382d87daa3c3799ad/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:23.380038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 909/1810] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:23.381201 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0ad01-a48b-448f-b32c-768e0144e690 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:23.386551 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd7affb-6a58-4297-ae91-a8307cf873d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22784ba66b5247b880f3bf88ae2a900e, user_id=08480a18e2f9491382d87daa3c3799ad, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:23.390336 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd7affb-6a58-4297-ae91-a8307cf873d1 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:23.397430 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0ad01-a48b-448f-b32c-768e0144e690 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:23.408165 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf96ac-1f51-4d85-8e3e-b1da1e5e25ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:23.408937 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf96ac-1f51-4d85-8e3e-b1da1e5e25ca admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:23.409292 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf96ac-1f51-4d85-8e3e-b1da1e5e25ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:23.409670 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 902/1811] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:23 2022] PUT /identity/v3/projects/22784ba66b5247b880f3bf88ae2a900e/users/08480a18e2f9491382d87daa3c3799ad/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:23.412589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf96ac-1f51-4d85-8e3e-b1da1e5e25ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:23.412737 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd7affb-6a58-4297-ae91-a8307cf873d1 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:23.412824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bf96ac-1f51-4d85-8e3e-b1da1e5e25ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:23.413225 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bf96ac-1f51-4d85-8e3e-b1da1e5e25ca admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:23.449008 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 910/1812] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:09:23 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:23.454002 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf96ac-1f51-4d85-8e3e-b1da1e5e25ca admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:23.456207 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a1bf96ac-1f51-4d85-8e3e-b1da1e5e25ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:23.491232 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd7affb-6a58-4297-ae91-a8307cf873d1 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:23.535535 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf96ac-1f51-4d85-8e3e-b1da1e5e25ca admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:23.545155 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2811f-3e97-4b36-9f63-b7090e8f1b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:23.546034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2811f-3e97-4b36-9f63-b7090e8f1b51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:23.546422 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2811f-3e97-4b36-9f63-b7090e8f1b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:23.546804 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2811f-3e97-4b36-9f63-b7090e8f1b51 admin admin] PATH_INFO: `/v3/projects/e197a27e5f4d45b9986adad2b26b7e50` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:23.548067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b2811f-3e97-4b36-9f63-b7090e8f1b51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e197a27e5f4d45b9986adad2b26b7e50)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:23.550783 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b2811f-3e97-4b36-9f63-b7090e8f1b51 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:23.569539 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057e415-16c5-486d-9c01-2a9bb1edbf5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:23.571034 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057e415-16c5-486d-9c01-2a9bb1edbf5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:23.571516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057e415-16c5-486d-9c01-2a9bb1edbf5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:23.571991 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057e415-16c5-486d-9c01-2a9bb1edbf5e admin admin] PATH_INFO: `/v3/projects/b30a9f60ed954f00b8675e21a0d81b3d/users/1c7a6262d22c4abeb57cd51346cb691d/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:23.595088 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057e415-16c5-486d-9c01-2a9bb1edbf5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b30a9f60ed954f00b8675e21a0d81b3d, user_id=1c7a6262d22c4abeb57cd51346cb691d, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:23.599849 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057e415-16c5-486d-9c01-2a9bb1edbf5e admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:23.603524 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:23.617410 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 903/1813] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:23 2022] PUT /identity/v3/projects/b30a9f60ed954f00b8675e21a0d81b3d/users/1c7a6262d22c4abeb57cd51346cb691d/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:23.618768 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2d7e7-7dbe-49f8-b630-96dfb92fdd98 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:23.619361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2d7e7-7dbe-49f8-b630-96dfb92fdd98 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:23.619477 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2d7e7-7dbe-49f8-b630-96dfb92fdd98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:23.622062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-d5b2811f-3e97-4b36-9f63-b7090e8f1b51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e197a27e5f4d45b9986adad2b26b7e50', 'request_id': 'req-d5b2811f-3e97-4b36-9f63-b7090e8f1b51'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:23.631445 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-d5b2811f-3e97-4b36-9f63-b7090e8f1b51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e197a27e5f4d45b9986adad2b26b7e50', 'request_id': 'req-d5b2811f-3e97-4b36-9f63-b7090e8f1b51'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:23.638536 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-d5b2811f-3e97-4b36-9f63-b7090e8f1b51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e197a27e5f4d45b9986adad2b26b7e50', 'request_id': 'req-d5b2811f-3e97-4b36-9f63-b7090e8f1b51'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:23.650193 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-d5b2811f-3e97-4b36-9f63-b7090e8f1b51 admin admin] The token cache is being invalidate because project e197a27e5f4d45b9986adad2b26b7e50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:09:23.650592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-d5b2811f-3e97-4b36-9f63-b7090e8f1b51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:23.650962 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-ebf2d7e7-7dbe-49f8-b630-96dfb92fdd98 None None] MFA Rules not processed for user `08480a18e2f9491382d87daa3c3799ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:23.651247 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-d5b2811f-3e97-4b36-9f63-b7090e8f1b51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:23.652225 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ebf2d7e7-7dbe-49f8-b630-96dfb92fdd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:23.652776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 911/1814] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:09:23 2022] DELETE /identity/v3/projects/e197a27e5f4d45b9986adad2b26b7e50 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:23.653514 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2811f-3e97-4b36-9f63-b7090e8f1b51 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:23.654763 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d5b2811f-3e97-4b36-9f63-b7090e8f1b51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:23.696444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:23.722239 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 904/1815] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:23.722848 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2811f-3e97-4b36-9f63-b7090e8f1b51 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:23.723524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e9e95-77db-4ffc-8551-4c29c5a5ab0f None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:23.723806 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e9e95-77db-4ffc-8551-4c29c5a5ab0f None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:23.724080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e9e95-77db-4ffc-8551-4c29c5a5ab0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:23.753061 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-ff4e9e95-77db-4ffc-8551-4c29c5a5ab0f None None] MFA Rules not processed for user `1c7a6262d22c4abeb57cd51346cb691d`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:23.754769 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ff4e9e95-77db-4ffc-8551-4c29c5a5ab0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:23.787886 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ce047-94d6-405c-ac8d-eb6a7f234a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:23.788425 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ce047-94d6-405c-ac8d-eb6a7f234a62 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:23.788740 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ce047-94d6-405c-ac8d-eb6a7f234a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:23.789019 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ce047-94d6-405c-ac8d-eb6a7f234a62 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:23.789648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ce047-94d6-405c-ac8d-eb6a7f234a62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:23.791606 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ce047-94d6-405c-ac8d-eb6a7f234a62 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:23.814400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 912/1816] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:09:23 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:23.815856 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69bc48-f6eb-48cd-a78c-e400a153be9b None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:23.816147 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69bc48-f6eb-48cd-a78c-e400a153be9b None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:23.816425 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69bc48-f6eb-48cd-a78c-e400a153be9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:23.822918 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 905/1817] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:23.823764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e9e95-77db-4ffc-8551-4c29c5a5ab0f None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:23.825043 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ff4e9e95-77db-4ffc-8551-4c29c5a5ab0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:23.836622 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-5e69bc48-f6eb-48cd-a78c-e400a153be9b None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:23.837807 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-5e69bc48-f6eb-48cd-a78c-e400a153be9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:23.846387 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 913/1818] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:23.847186 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69bc48-f6eb-48cd-a78c-e400a153be9b None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:23.856529 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69bc48-f6eb-48cd-a78c-e400a153be9b None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:23.862213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba870d-a9d5-437f-b111-0f2a9c1b9476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:23.862812 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba870d-a9d5-437f-b111-0f2a9c1b9476 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:23.863089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba870d-a9d5-437f-b111-0f2a9c1b9476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:23.863360 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba870d-a9d5-437f-b111-0f2a9c1b9476 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:23.863986 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba870d-a9d5-437f-b111-0f2a9c1b9476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:23.865590 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ba870d-a9d5-437f-b111-0f2a9c1b9476 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:23.875873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 914/1819] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:23.876693 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba870d-a9d5-437f-b111-0f2a9c1b9476 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:23.878271 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-e1ba870d-a9d5-437f-b111-0f2a9c1b9476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:23.884920 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e9e95-77db-4ffc-8551-4c29c5a5ab0f None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:23.888677 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e7301-7041-451f-a29a-12437c323232 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:23.889283 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e7301-7041-451f-a29a-12437c323232 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:23.889567 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e7301-7041-451f-a29a-12437c323232 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:23.890338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e7301-7041-451f-a29a-12437c323232 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:23.892252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba870d-a9d5-437f-b111-0f2a9c1b9476 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:23.896848 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a839d-b26b-4b72-aee5-dbb45476c6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:23.897439 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a839d-b26b-4b72-aee5-dbb45476c6ac admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:23.897716 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a839d-b26b-4b72-aee5-dbb45476c6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:23.897988 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a839d-b26b-4b72-aee5-dbb45476c6ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:23.898609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a839d-b26b-4b72-aee5-dbb45476c6ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:23.898912 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e7301-7041-451f-a29a-12437c323232 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:23.900351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e7301-7041-451f-a29a-12437c323232 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:23.900457 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a839d-b26b-4b72-aee5-dbb45476c6ac admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:23.909527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 906/1820] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:09:23.911836 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f4846-7c25-45ac-8be4-1f6d39070e4a None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:23.912128 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f4846-7c25-45ac-8be4-1f6d39070e4a None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:23.912404 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f4846-7c25-45ac-8be4-1f6d39070e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:23.930508 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 915/1821] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:09:23 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:23.931349 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a839d-b26b-4b72-aee5-dbb45476c6ac admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:23.936105 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-a59f4846-7c25-45ac-8be4-1f6d39070e4a None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:23.938864 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a59f4846-7c25-45ac-8be4-1f6d39070e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:23.951050 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:24.005587 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a839d-b26b-4b72-aee5-dbb45476c6ac admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:24.006877 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 907/1822] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:24.007700 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f4846-7c25-45ac-8be4-1f6d39070e4a None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:24.013703 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cc9bd-bbc5-4192-96c2-719e7e8ed5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:24.014604 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cc9bd-bbc5-4192-96c2-719e7e8ed5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:24.015597 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cc9bd-bbc5-4192-96c2-719e7e8ed5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:24.015933 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cc9bd-bbc5-4192-96c2-719e7e8ed5d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:24.016648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716cc9bd-bbc5-4192-96c2-719e7e8ed5d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:24.018847 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716cc9bd-bbc5-4192-96c2-719e7e8ed5d4 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:24.020768 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f4846-7c25-45ac-8be4-1f6d39070e4a None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:24.025508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52494895-ead4-4c17-bd42-46a02c073fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:24.026107 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52494895-ead4-4c17-bd42-46a02c073fd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:24.026389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52494895-ead4-4c17-bd42-46a02c073fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:24.026664 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52494895-ead4-4c17-bd42-46a02c073fd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:24.027297 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52494895-ead4-4c17-bd42-46a02c073fd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:24.029225 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52494895-ead4-4c17-bd42-46a02c073fd1 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:24.033805 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 916/1823] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:24.033805 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cc9bd-bbc5-4192-96c2-719e7e8ed5d4 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:24.045204 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cc9bd-bbc5-4192-96c2-719e7e8ed5d4 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:24.048775 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:24.051118 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5626f7c-d1f8-4004-bd61-5e2fa511bd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:24.051938 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5626f7c-d1f8-4004-bd61-5e2fa511bd6a admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:24.052309 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5626f7c-d1f8-4004-bd61-5e2fa511bd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:24.052716 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5626f7c-d1f8-4004-bd61-5e2fa511bd6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:24.053440 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5626f7c-d1f8-4004-bd61-5e2fa511bd6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:09:24.053817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5626f7c-d1f8-4004-bd61-5e2fa511bd6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:24.055487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5626f7c-d1f8-4004-bd61-5e2fa511bd6a admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:24.057652 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 908/1824] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:09:24 2022] POST /identity/v3/users => generated 299 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:24.058834 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52494895-ead4-4c17-bd42-46a02c073fd1 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:24.072371 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 917/1825] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:09:24.072863 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52494895-ead4-4c17-bd42-46a02c073fd1 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:24.073307 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5626f7c-d1f8-4004-bd61-5e2fa511bd6a admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:24.080857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a2770-7e77-409b-a782-db90fd9290c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:24.081533 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a2770-7e77-409b-a782-db90fd9290c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:24.081828 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a2770-7e77-409b-a782-db90fd9290c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:24.082081 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a2770-7e77-409b-a782-db90fd9290c1 admin admin] PATH_INFO: `/v3/projects/fa872cc55f5c4953a1f469934f3ed115/users/2e15fcb714be486e99d0905b3d88fc6f/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:24.083863 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5626f7c-d1f8-4004-bd61-5e2fa511bd6a admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:24.089178 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db283f-a64d-4be4-964d-af34c25dc791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:24.089791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db283f-a64d-4be4-964d-af34c25dc791 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:24.090135 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db283f-a64d-4be4-964d-af34c25dc791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:24.090359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db283f-a64d-4be4-964d-af34c25dc791 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:24.091115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db283f-a64d-4be4-964d-af34c25dc791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:24.092907 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db283f-a64d-4be4-964d-af34c25dc791 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:24.104452 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 918/1826] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:24.105301 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db283f-a64d-4be4-964d-af34c25dc791 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:24.112079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28a2770-7e77-409b-a782-db90fd9290c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa872cc55f5c4953a1f469934f3ed115, user_id=2e15fcb714be486e99d0905b3d88fc6f, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:24.116854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db283f-a64d-4be4-964d-af34c25dc791 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:24.117804 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28a2770-7e77-409b-a782-db90fd9290c1 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:24.122804 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ea913-8371-43a2-bbc2-f653975d6c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:24.123576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ea913-8371-43a2-bbc2-f653975d6c32 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:24.124075 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ea913-8371-43a2-bbc2-f653975d6c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:24.124950 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ea913-8371-43a2-bbc2-f653975d6c32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:24.125822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ea913-8371-43a2-bbc2-f653975d6c32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:24.127816 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ea913-8371-43a2-bbc2-f653975d6c32 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:24.138583 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 909/1827] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:24 2022] PUT /identity/v3/projects/fa872cc55f5c4953a1f469934f3ed115/users/2e15fcb714be486e99d0905b3d88fc6f/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:24.139397 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a2770-7e77-409b-a782-db90fd9290c1 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:24.163169 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 919/1828] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:09:24 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:24.164454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b222e-342c-477d-81a9-e87f70007b53 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:24.164789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b222e-342c-477d-81a9-e87f70007b53 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:24.165071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4b222e-342c-477d-81a9-e87f70007b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:24.200556 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-de4b222e-342c-477d-81a9-e87f70007b53 None None] MFA Rules not processed for user `2e15fcb714be486e99d0905b3d88fc6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:24.201868 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-de4b222e-342c-477d-81a9-e87f70007b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:24.259878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a2770-7e77-409b-a782-db90fd9290c1 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:24.319517 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 920/1829] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:24.320186 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b222e-342c-477d-81a9-e87f70007b53 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:24.349697 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d47eb-bc24-4068-9545-45c1aed3865d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:24.350459 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d47eb-bc24-4068-9545-45c1aed3865d admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:24.350757 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d47eb-bc24-4068-9545-45c1aed3865d admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:24.351040 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d47eb-bc24-4068-9545-45c1aed3865d admin admin] PATH_INFO: `/v3/projects/3f51ac081e72406ca83e055f039136d2/users/3727e43927674c2cb5b6f33ecd464ccd/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:24.368326 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d47eb-bc24-4068-9545-45c1aed3865d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f51ac081e72406ca83e055f039136d2, user_id=3727e43927674c2cb5b6f33ecd464ccd, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:24.371152 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d47eb-bc24-4068-9545-45c1aed3865d admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:24.386658 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 910/1830] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:24 2022] PUT /identity/v3/projects/3f51ac081e72406ca83e055f039136d2/users/3727e43927674c2cb5b6f33ecd464ccd/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:24.387542 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d47eb-bc24-4068-9545-45c1aed3865d admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:24.389065 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:24.397338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4b222e-342c-477d-81a9-e87f70007b53 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:24.456585 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d47eb-bc24-4068-9545-45c1aed3865d admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:24.461617 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ca683-fd8e-41a7-a9ef-cf463c4e829a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:24.462236 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ca683-fd8e-41a7-a9ef-cf463c4e829a admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:24.462522 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ca683-fd8e-41a7-a9ef-cf463c4e829a admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:24.462802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ca683-fd8e-41a7-a9ef-cf463c4e829a admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:24.463449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1ca683-fd8e-41a7-a9ef-cf463c4e829a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:24.465426 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1ca683-fd8e-41a7-a9ef-cf463c4e829a admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:24.471162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70826d44-e994-418e-b41c-d5aae4808772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:24.471807 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70826d44-e994-418e-b41c-d5aae4808772 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:24.472090 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70826d44-e994-418e-b41c-d5aae4808772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:24.472370 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70826d44-e994-418e-b41c-d5aae4808772 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:24.473035 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70826d44-e994-418e-b41c-d5aae4808772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:24.474942 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70826d44-e994-418e-b41c-d5aae4808772 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:24.477753 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:24.501174 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 911/1832] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:09:24 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:24.501382 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 921/1832] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:09:24 2022] POST /identity/v3/users => generated 292 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:24.501826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ca683-fd8e-41a7-a9ef-cf463c4e829a admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:24.504994 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70826d44-e994-418e-b41c-d5aae4808772 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:24.514904 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70826d44-e994-418e-b41c-d5aae4808772 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:24.581278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ca683-fd8e-41a7-a9ef-cf463c4e829a admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:24.581804 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599997a-33a8-4aa1-bd15-4b85afe60a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:24.582239 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599997a-33a8-4aa1-bd15-4b85afe60a3a admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:24.582563 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599997a-33a8-4aa1-bd15-4b85afe60a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:24.582842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599997a-33a8-4aa1-bd15-4b85afe60a3a admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:24.583479 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599997a-33a8-4aa1-bd15-4b85afe60a3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:24.585475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599997a-33a8-4aa1-bd15-4b85afe60a3a admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:24.586228 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a3de2-3f94-4412-a787-8318380187b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:24.586825 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a3de2-3f94-4412-a787-8318380187b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:24.587123 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a3de2-3f94-4412-a787-8318380187b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:24.587394 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a3de2-3f94-4412-a787-8318380187b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:24.588024 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a3de2-3f94-4412-a787-8318380187b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:24.589600 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a3de2-3f94-4412-a787-8318380187b7 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:24.604773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 912/1833] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:24.604773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a3de2-3f94-4412-a787-8318380187b7 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:24.617690 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 922/1834] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:09:24 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:09:24.617690 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599997a-33a8-4aa1-bd15-4b85afe60a3a admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:24.618271 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a3de2-3f94-4412-a787-8318380187b7 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:24.623965 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fae063-7d3f-4240-b2fd-acd8c369f781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:24.625407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fae063-7d3f-4240-b2fd-acd8c369f781 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:24.625684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fae063-7d3f-4240-b2fd-acd8c369f781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:24.625957 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fae063-7d3f-4240-b2fd-acd8c369f781 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:24.626590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fae063-7d3f-4240-b2fd-acd8c369f781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:24.628157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fae063-7d3f-4240-b2fd-acd8c369f781 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:24.630737 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599997a-33a8-4aa1-bd15-4b85afe60a3a admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:24.635964 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6091a-6ecc-4d6a-8980-f1023d1675b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:24.636815 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6091a-6ecc-4d6a-8980-f1023d1675b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:24.637163 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6091a-6ecc-4d6a-8980-f1023d1675b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:24.637518 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6091a-6ecc-4d6a-8980-f1023d1675b6 admin admin] PATH_INFO: `/v3/projects/3f51ac081e72406ca83e055f039136d2/users/3727e43927674c2cb5b6f33ecd464ccd/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:24.639485 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca6091a-6ecc-4d6a-8980-f1023d1675b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f51ac081e72406ca83e055f039136d2, user_id=3727e43927674c2cb5b6f33ecd464ccd, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:24.642473 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 913/1835] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:24.643484 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca6091a-6ecc-4d6a-8980-f1023d1675b6 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:24.643623 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fae063-7d3f-4240-b2fd-acd8c369f781 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:24.654953 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fae063-7d3f-4240-b2fd-acd8c369f781 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:24.663787 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 923/1836] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:24 2022] PUT /identity/v3/projects/3f51ac081e72406ca83e055f039136d2/users/3727e43927674c2cb5b6f33ecd464ccd/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:24.665062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6091a-6ecc-4d6a-8980-f1023d1675b6 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:24.714677 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe6e9c-f487-4c89-b008-9e211b161685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:24.715290 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe6e9c-f487-4c89-b008-9e211b161685 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:24.715575 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe6e9c-f487-4c89-b008-9e211b161685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:24.715853 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe6e9c-f487-4c89-b008-9e211b161685 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:24.716509 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe6e9c-f487-4c89-b008-9e211b161685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:24.718254 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe6e9c-f487-4c89-b008-9e211b161685 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:24.731749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 914/1837] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:24.731749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e30acb-4dd6-4e68-b531-45e0d60bae65 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:24.731749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e30acb-4dd6-4e68-b531-45e0d60bae65 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:24.731749 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e30acb-4dd6-4e68-b531-45e0d60bae65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:24.737473 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6091a-6ecc-4d6a-8980-f1023d1675b6 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:24.743013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1a9d1-abaa-422f-9edb-7aeb71029b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:24.743624 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1a9d1-abaa-422f-9edb-7aeb71029b1b admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:24.743911 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1a9d1-abaa-422f-9edb-7aeb71029b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:24.744660 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d1a9d1-abaa-422f-9edb-7aeb71029b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:24.745314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1a9d1-abaa-422f-9edb-7aeb71029b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:24.746921 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d1a9d1-abaa-422f-9edb-7aeb71029b1b admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:24.762384 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 924/1838] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:24.762683 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-e0e30acb-4dd6-4e68-b531-45e0d60bae65 None None] MFA Rules not processed for user `3727e43927674c2cb5b6f33ecd464ccd`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:24.762804 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1a9d1-abaa-422f-9edb-7aeb71029b1b admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:24.762885 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e0e30acb-4dd6-4e68-b531-45e0d60bae65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:24.767247 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:24.771265 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d1a9d1-abaa-422f-9edb-7aeb71029b1b admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:24.777252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578133e-8f0c-4fcb-bb7e-312b0df4cdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:24.777903 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578133e-8f0c-4fcb-bb7e-312b0df4cdc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:24.778182 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578133e-8f0c-4fcb-bb7e-312b0df4cdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:24.778461 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578133e-8f0c-4fcb-bb7e-312b0df4cdc3 admin admin] PATH_INFO: `/v3/projects/57a110d5b8654694b6f8e21085c7b287/users/e33c048e2d1842348db0a6c8a7707cd2/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:24.793290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578133e-8f0c-4fcb-bb7e-312b0df4cdc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57a110d5b8654694b6f8e21085c7b287, user_id=e33c048e2d1842348db0a6c8a7707cd2, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:24.796699 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578133e-8f0c-4fcb-bb7e-312b0df4cdc3 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:24.809986 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 925/1839] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:24 2022] PUT /identity/v3/projects/57a110d5b8654694b6f8e21085c7b287/users/e33c048e2d1842348db0a6c8a7707cd2/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:24.810707 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578133e-8f0c-4fcb-bb7e-312b0df4cdc3 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:24.827260 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:24.838513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 915/1840] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:24 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:24.839497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e30acb-4dd6-4e68-b531-45e0d60bae65 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:24.851076 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e30acb-4dd6-4e68-b531-45e0d60bae65 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:24.876078 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578133e-8f0c-4fcb-bb7e-312b0df4cdc3 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:24.880762 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8f9da-136a-43eb-84fe-4ecbf9bd76d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:24.881443 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8f9da-136a-43eb-84fe-4ecbf9bd76d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:24.881759 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8f9da-136a-43eb-84fe-4ecbf9bd76d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:24.882031 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8f9da-136a-43eb-84fe-4ecbf9bd76d6 admin admin] PATH_INFO: `/v3/projects/290eb4a266224e7ca20ae34c9196b685/users/2e0cf3d88ff14199b098b0a7dff2c52b/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:24.897048 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8f9da-136a-43eb-84fe-4ecbf9bd76d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=290eb4a266224e7ca20ae34c9196b685, user_id=2e0cf3d88ff14199b098b0a7dff2c52b, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:24.899923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8f9da-136a-43eb-84fe-4ecbf9bd76d6 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:24.928981 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 926/1841] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:24 2022] PUT /identity/v3/projects/290eb4a266224e7ca20ae34c9196b685/users/2e0cf3d88ff14199b098b0a7dff2c52b/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:24.929891 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8f9da-136a-43eb-84fe-4ecbf9bd76d6 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:24.947242 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729a154-64e0-4422-a39a-e852471c961a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:24.947866 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729a154-64e0-4422-a39a-e852471c961a admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:24.948160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729a154-64e0-4422-a39a-e852471c961a admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:24.948439 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729a154-64e0-4422-a39a-e852471c961a admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:24.949130 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729a154-64e0-4422-a39a-e852471c961a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:24.950741 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729a154-64e0-4422-a39a-e852471c961a admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:24.964161 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 916/1842] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:24.964161 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124678c-64d9-4b3c-9bd3-42c2165e4b75 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:24.964161 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124678c-64d9-4b3c-9bd3-42c2165e4b75 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:24.964161 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124678c-64d9-4b3c-9bd3-42c2165e4b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:24.993495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-f124678c-64d9-4b3c-9bd3-42c2165e4b75 None None] MFA Rules not processed for user `2e0cf3d88ff14199b098b0a7dff2c52b`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:24.994735 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f124678c-64d9-4b3c-9bd3-42c2165e4b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:24.996086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8f9da-136a-43eb-84fe-4ecbf9bd76d6 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:25.000105 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e05e7-e4a5-4ee9-9b85-57d469912d65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:25.000844 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e05e7-e4a5-4ee9-9b85-57d469912d65 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:25.001121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e05e7-e4a5-4ee9-9b85-57d469912d65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:25.001396 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e05e7-e4a5-4ee9-9b85-57d469912d65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:25.062849 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 917/1843] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:25.063806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e05e7-e4a5-4ee9-9b85-57d469912d65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:25.064076 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124678c-64d9-4b3c-9bd3-42c2165e4b75 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:25.065298 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e05e7-e4a5-4ee9-9b85-57d469912d65 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:25.075412 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 927/1844] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:24 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:09:25.076290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e05e7-e4a5-4ee9-9b85-57d469912d65 service cinder] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:25.090036 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:25.155232 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e05e7-e4a5-4ee9-9b85-57d469912d65 service cinder] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:25.160803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124678c-64d9-4b3c-9bd3-42c2165e4b75 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:25.162174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2b411-9e08-4e5c-b131-359dce951f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:25.162856 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2b411-9e08-4e5c-b131-359dce951f92 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:25.163144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2b411-9e08-4e5c-b131-359dce951f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:25.163425 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2b411-9e08-4e5c-b131-359dce951f92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:25.164092 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2b411-9e08-4e5c-b131-359dce951f92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:25.165791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4918f6-03d3-4b1c-bbe7-ae3a6d7adf04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:25.166008 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2b411-9e08-4e5c-b131-359dce951f92 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:25.166447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4918f6-03d3-4b1c-bbe7-ae3a6d7adf04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:25.166728 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4918f6-03d3-4b1c-bbe7-ae3a6d7adf04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:25.167071 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4918f6-03d3-4b1c-bbe7-ae3a6d7adf04 admin admin] PATH_INFO: `/v3/projects/57a110d5b8654694b6f8e21085c7b287/users/e33c048e2d1842348db0a6c8a7707cd2/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:25.169012 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4918f6-03d3-4b1c-bbe7-ae3a6d7adf04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57a110d5b8654694b6f8e21085c7b287, user_id=e33c048e2d1842348db0a6c8a7707cd2, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:25.172425 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4918f6-03d3-4b1c-bbe7-ae3a6d7adf04 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:25.185405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 918/1845] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:25 2022] PUT /identity/v3/projects/57a110d5b8654694b6f8e21085c7b287/users/e33c048e2d1842348db0a6c8a7707cd2/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:25.196803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5902235-1999-4d59-bf22-26bb7fdd1760 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:25.197198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5902235-1999-4d59-bf22-26bb7fdd1760 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:25.197415 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5902235-1999-4d59-bf22-26bb7fdd1760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:25.204544 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 928/1846] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:09:25 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:25.205551 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2b411-9e08-4e5c-b131-359dce951f92 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:25.235588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-f5902235-1999-4d59-bf22-26bb7fdd1760 None None] MFA Rules not processed for user `e33c048e2d1842348db0a6c8a7707cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:25.236840 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f5902235-1999-4d59-bf22-26bb7fdd1760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:25.242613 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:25.281269 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2b411-9e08-4e5c-b131-359dce951f92 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:25.288046 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc14a2-c980-4b7c-a373-be2a94276492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:25.288665 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc14a2-c980-4b7c-a373-be2a94276492 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:25.289036 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc14a2-c980-4b7c-a373-be2a94276492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:25.289376 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc14a2-c980-4b7c-a373-be2a94276492 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:25.290158 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc14a2-c980-4b7c-a373-be2a94276492 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:25.292479 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc14a2-c980-4b7c-a373-be2a94276492 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:25.332970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 929/1847] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:09:25 2022] POST /identity/v3/users => generated 297 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:25.335845 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc14a2-c980-4b7c-a373-be2a94276492 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:25.353470 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc14a2-c980-4b7c-a373-be2a94276492 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:25.361182 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612b230-5223-4757-8d85-237d5da6cd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:25.362103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612b230-5223-4757-8d85-237d5da6cd78 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:25.362532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612b230-5223-4757-8d85-237d5da6cd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:25.363116 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612b230-5223-4757-8d85-237d5da6cd78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:25.364059 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612b230-5223-4757-8d85-237d5da6cd78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:25.365340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 919/1848] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:09:25.366273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612b230-5223-4757-8d85-237d5da6cd78 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:25.377763 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5902235-1999-4d59-bf22-26bb7fdd1760 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:25.380164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 930/1849] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:25.386252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612b230-5223-4757-8d85-237d5da6cd78 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:25.397529 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612b230-5223-4757-8d85-237d5da6cd78 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:25.403102 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce18c71-c63e-48c6-a88b-73803cde4409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:25.404148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce18c71-c63e-48c6-a88b-73803cde4409 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:25.404675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce18c71-c63e-48c6-a88b-73803cde4409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:25.405588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce18c71-c63e-48c6-a88b-73803cde4409 admin admin] PATH_INFO: `/v3/projects/3a983949349947df9fd050dd3f5f6d45/users/4e0f096d7a724d079869c0587b2adf55/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:25.428285 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce18c71-c63e-48c6-a88b-73803cde4409 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a983949349947df9fd050dd3f5f6d45, user_id=4e0f096d7a724d079869c0587b2adf55, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:25.431367 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce18c71-c63e-48c6-a88b-73803cde4409 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:25.448888 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 931/1850] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:25 2022] PUT /identity/v3/projects/3a983949349947df9fd050dd3f5f6d45/users/4e0f096d7a724d079869c0587b2adf55/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:09:25.452652 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce18c71-c63e-48c6-a88b-73803cde4409 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:25.455579 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5902235-1999-4d59-bf22-26bb7fdd1760 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:25.507767 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:25.511993 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4ce17-570e-46da-bf0b-2b0e57334f9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:25.512561 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4ce17-570e-46da-bf0b-2b0e57334f9d service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:25.512849 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4ce17-570e-46da-bf0b-2b0e57334f9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:25.513126 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4ce17-570e-46da-bf0b-2b0e57334f9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:25.515976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-c1f4ce17-570e-46da-bf0b-2b0e57334f9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:25.524909 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce18c71-c63e-48c6-a88b-73803cde4409 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:25.532306 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec12c05-7178-43e7-afc6-a5359509f93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:25.532968 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec12c05-7178-43e7-afc6-a5359509f93d admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:25.533257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec12c05-7178-43e7-afc6-a5359509f93d admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:25.533536 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec12c05-7178-43e7-afc6-a5359509f93d admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:25.534265 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec12c05-7178-43e7-afc6-a5359509f93d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:25.535962 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec12c05-7178-43e7-afc6-a5359509f93d admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:25.549875 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 932/1851] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:09:25.554116 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec12c05-7178-43e7-afc6-a5359509f93d admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:25.566196 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec12c05-7178-43e7-afc6-a5359509f93d admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:25.573923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0dc94-b7d8-49e9-83a7-87359094ce72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:25.574625 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0dc94-b7d8-49e9-83a7-87359094ce72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:25.574909 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0dc94-b7d8-49e9-83a7-87359094ce72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:25.575185 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0dc94-b7d8-49e9-83a7-87359094ce72 admin admin] PATH_INFO: `/v3/projects/3a983949349947df9fd050dd3f5f6d45/users/4e0f096d7a724d079869c0587b2adf55/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:25.581072 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0dc94-b7d8-49e9-83a7-87359094ce72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a983949349947df9fd050dd3f5f6d45, user_id=4e0f096d7a724d079869c0587b2adf55, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:25.583825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0dc94-b7d8-49e9-83a7-87359094ce72 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:25.590614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4ce17-570e-46da-bf0b-2b0e57334f9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:25.593466 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4ce17-570e-46da-bf0b-2b0e57334f9d service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:25.603072 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 933/1852] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:25 2022] PUT /identity/v3/projects/3a983949349947df9fd050dd3f5f6d45/users/4e0f096d7a724d079869c0587b2adf55/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:25.605806 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 920/1853] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:09:25.622689 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c9aef-f8d7-4c1c-b33a-bdb84e270691 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:25.623048 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c9aef-f8d7-4c1c-b33a-bdb84e270691 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:25.623393 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c9aef-f8d7-4c1c-b33a-bdb84e270691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:25.674099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-2b2c9aef-f8d7-4c1c-b33a-bdb84e270691 None None] MFA Rules not processed for user `4e0f096d7a724d079869c0587b2adf55`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:25.675434 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2b2c9aef-f8d7-4c1c-b33a-bdb84e270691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:25.781343 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 934/1854] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:25.832793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4ce17-570e-46da-bf0b-2b0e57334f9d service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:25.834853 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-c1f4ce17-570e-46da-bf0b-2b0e57334f9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:25.848136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4ce17-570e-46da-bf0b-2b0e57334f9d service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:25.856874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:25.902856 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc50c96-82b4-4f3a-b954-540298061476 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:25.903510 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc50c96-82b4-4f3a-b954-540298061476 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:25.903797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc50c96-82b4-4f3a-b954-540298061476 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:25.904100 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc50c96-82b4-4f3a-b954-540298061476 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:25.915480 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc50c96-82b4-4f3a-b954-540298061476 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:25.917314 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc50c96-82b4-4f3a-b954-540298061476 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:25.929578 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 921/1855] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:09:26.100560 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c9aef-f8d7-4c1c-b33a-bdb84e270691 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:26.111393 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c9aef-f8d7-4c1c-b33a-bdb84e270691 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:26.116895 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99b4e4-e781-4a72-92b9-8013a8a85b0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:26.117606 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99b4e4-e781-4a72-92b9-8013a8a85b0e service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:26.117957 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99b4e4-e781-4a72-92b9-8013a8a85b0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:26.118307 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99b4e4-e781-4a72-92b9-8013a8a85b0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:26.177314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b99b4e4-e781-4a72-92b9-8013a8a85b0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:26.178827 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b99b4e4-e781-4a72-92b9-8013a8a85b0e service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:26.190928 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 935/1856] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:09:26.327563 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc50c96-82b4-4f3a-b954-540298061476 service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:26.424778 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc50c96-82b4-4f3a-b954-540298061476 service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:26.429339 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff0b0e-792f-47f9-9845-983ab0e6bf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:26.430256 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff0b0e-792f-47f9-9845-983ab0e6bf3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:26.430602 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff0b0e-792f-47f9-9845-983ab0e6bf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:26.430941 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff0b0e-792f-47f9-9845-983ab0e6bf3e admin admin] PATH_INFO: `/v3/users/1c7a6262d22c4abeb57cd51346cb691d` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:26.432135 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff0b0e-792f-47f9-9845-983ab0e6bf3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c7a6262d22c4abeb57cd51346cb691d)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:26.434105 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff0b0e-792f-47f9-9845-983ab0e6bf3e admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:26.485908 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f5ff0b0e-792f-47f9-9845-983ab0e6bf3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c7a6262d22c4abeb57cd51346cb691d'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:26.493543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f5ff0b0e-792f-47f9-9845-983ab0e6bf3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c7a6262d22c4abeb57cd51346cb691d'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:26.509437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f5ff0b0e-792f-47f9-9845-983ab0e6bf3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c7a6262d22c4abeb57cd51346cb691d'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:26.510141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f5ff0b0e-792f-47f9-9845-983ab0e6bf3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c7a6262d22c4abeb57cd51346cb691d'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:26.510747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f5ff0b0e-792f-47f9-9845-983ab0e6bf3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c7a6262d22c4abeb57cd51346cb691d'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:26.533459 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 922/1857] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:09:26 2022] DELETE /identity/v3/users/1c7a6262d22c4abeb57cd51346cb691d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:26.537666 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c27f19-7f3e-4130-a5ce-278c05fd163c None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:26.537959 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c27f19-7f3e-4130-a5ce-278c05fd163c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:26.538241 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c27f19-7f3e-4130-a5ce-278c05fd163c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:26.546077 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff0b0e-792f-47f9-9845-983ab0e6bf3e admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:26.547504 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f5ff0b0e-792f-47f9-9845-983ab0e6bf3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:26.561041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-66c27f19-7f3e-4130-a5ce-278c05fd163c None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:26.562990 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-66c27f19-7f3e-4130-a5ce-278c05fd163c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:26.584561 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:26.615975 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff0b0e-792f-47f9-9845-983ab0e6bf3e admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:26.618909 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:26.622066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a57982-f9cd-4fa0-8adf-6b7ae743809f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:26.622711 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a57982-f9cd-4fa0-8adf-6b7ae743809f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:26.623005 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a57982-f9cd-4fa0-8adf-6b7ae743809f admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:26.623274 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a57982-f9cd-4fa0-8adf-6b7ae743809f admin admin] PATH_INFO: `/v3/users/2e0cf3d88ff14199b098b0a7dff2c52b` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:26.625045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a57982-f9cd-4fa0-8adf-6b7ae743809f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e0cf3d88ff14199b098b0a7dff2c52b)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:26.627048 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a57982-f9cd-4fa0-8adf-6b7ae743809f admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:26.632050 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 936/1858] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:26.642409 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c27f19-7f3e-4130-a5ce-278c05fd163c None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:26.644089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-66c27f19-7f3e-4130-a5ce-278c05fd163c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:26.692420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c2a57982-f9cd-4fa0-8adf-6b7ae743809f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e0cf3d88ff14199b098b0a7dff2c52b'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:26.706961 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c2a57982-f9cd-4fa0-8adf-6b7ae743809f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e0cf3d88ff14199b098b0a7dff2c52b'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:26.725629 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c2a57982-f9cd-4fa0-8adf-6b7ae743809f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e0cf3d88ff14199b098b0a7dff2c52b'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:26.726478 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c2a57982-f9cd-4fa0-8adf-6b7ae743809f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e0cf3d88ff14199b098b0a7dff2c52b'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:26.726588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c27f19-7f3e-4130-a5ce-278c05fd163c None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:26.727404 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c2a57982-f9cd-4fa0-8adf-6b7ae743809f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e0cf3d88ff14199b098b0a7dff2c52b'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:26.730395 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16080760-a7e9-4b68-b051-e4ccebc40366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:26.731012 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16080760-a7e9-4b68-b051-e4ccebc40366 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:26.731294 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16080760-a7e9-4b68-b051-e4ccebc40366 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:26.731569 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16080760-a7e9-4b68-b051-e4ccebc40366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:26.734440 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-16080760-a7e9-4b68-b051-e4ccebc40366 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:26.745137 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 923/1859] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:09:26 2022] DELETE /identity/v3/users/2e0cf3d88ff14199b098b0a7dff2c52b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:09:26.749406 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ca884-e264-4505-a715-26ed217c4c4a None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:26.750110 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ca884-e264-4505-a715-26ed217c4c4a None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:26.750219 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16080760-a7e9-4b68-b051-e4ccebc40366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:26.750691 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ca884-e264-4505-a715-26ed217c4c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:26.751752 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16080760-a7e9-4b68-b051-e4ccebc40366 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:26.761846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 937/1860] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:09:26.772719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1c4ca884-e264-4505-a715-26ed217c4c4a None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:26.773949 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1c4ca884-e264-4505-a715-26ed217c4c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:26.840169 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 924/1861] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:09:26.851655 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16080760-a7e9-4b68-b051-e4ccebc40366 service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:26.853945 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-16080760-a7e9-4b68-b051-e4ccebc40366 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:26.869185 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16080760-a7e9-4b68-b051-e4ccebc40366 service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:26.937695 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c3b1f-4611-4554-8f71-4dacb9b2348a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:26.938190 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c3b1f-4611-4554-8f71-4dacb9b2348a service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:26.938546 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c3b1f-4611-4554-8f71-4dacb9b2348a service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:26.938895 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c3b1f-4611-4554-8f71-4dacb9b2348a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:26.951865 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c3b1f-4611-4554-8f71-4dacb9b2348a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:26.954189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c3b1f-4611-4554-8f71-4dacb9b2348a service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:26.965274 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 938/1862] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:09:27.074027 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ca884-e264-4505-a715-26ed217c4c4a None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:27.076906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1c4ca884-e264-4505-a715-26ed217c4c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:27.094690 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ca884-e264-4505-a715-26ed217c4c4a None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:27.099650 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f63841-edaf-4298-bc1b-d389dfbced63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:27.100309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f63841-edaf-4298-bc1b-d389dfbced63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:27.100622 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f63841-edaf-4298-bc1b-d389dfbced63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:27.104629 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f63841-edaf-4298-bc1b-d389dfbced63 admin admin] PATH_INFO: `/v3/projects/b30a9f60ed954f00b8675e21a0d81b3d` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:27.111026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f63841-edaf-4298-bc1b-d389dfbced63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b30a9f60ed954f00b8675e21a0d81b3d)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:27.122847 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f63841-edaf-4298-bc1b-d389dfbced63 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:27.177096 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:27.178893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-02f63841-edaf-4298-bc1b-d389dfbced63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b30a9f60ed954f00b8675e21a0d81b3d', 'request_id': 'req-02f63841-edaf-4298-bc1b-d389dfbced63'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.185844 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-02f63841-edaf-4298-bc1b-d389dfbced63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b30a9f60ed954f00b8675e21a0d81b3d', 'request_id': 'req-02f63841-edaf-4298-bc1b-d389dfbced63'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.197204 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-02f63841-edaf-4298-bc1b-d389dfbced63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b30a9f60ed954f00b8675e21a0d81b3d', 'request_id': 'req-02f63841-edaf-4298-bc1b-d389dfbced63'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.210028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-02f63841-edaf-4298-bc1b-d389dfbced63 admin admin] The token cache is being invalidate because project b30a9f60ed954f00b8675e21a0d81b3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:09:27.210322 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-02f63841-edaf-4298-bc1b-d389dfbced63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.210964 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-02f63841-edaf-4298-bc1b-d389dfbced63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.212773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 925/1863] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:09:27 2022] DELETE /identity/v3/projects/b30a9f60ed954f00b8675e21a0d81b3d => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:27.216563 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c3b1f-4611-4554-8f71-4dacb9b2348a service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:27.227623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c3b1f-4611-4554-8f71-4dacb9b2348a service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:27.289239 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba19ab-823e-4925-965c-8cdfdc7bfc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:27.290009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba19ab-823e-4925-965c-8cdfdc7bfc33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:27.290293 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba19ab-823e-4925-965c-8cdfdc7bfc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:27.290572 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba19ab-823e-4925-965c-8cdfdc7bfc33 admin admin] PATH_INFO: `/v3/users/3727e43927674c2cb5b6f33ecd464ccd` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:27.291691 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba19ab-823e-4925-965c-8cdfdc7bfc33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3727e43927674c2cb5b6f33ecd464ccd)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:27.293645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba19ab-823e-4925-965c-8cdfdc7bfc33 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:27.304298 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f63841-edaf-4298-bc1b-d389dfbced63 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:27.306237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-02f63841-edaf-4298-bc1b-d389dfbced63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:27.337374 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f63841-edaf-4298-bc1b-d389dfbced63 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:27.342249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d20845-1148-4396-9bb5-cbf69c99f631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:27.343514 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d20845-1148-4396-9bb5-cbf69c99f631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:27.344049 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d20845-1148-4396-9bb5-cbf69c99f631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:27.344424 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d20845-1148-4396-9bb5-cbf69c99f631 admin admin] PATH_INFO: `/v3/projects/290eb4a266224e7ca20ae34c9196b685` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:27.345475 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d20845-1148-4396-9bb5-cbf69c99f631 admin admin] RBAC: Authorizing `identity:delete_project(project_id=290eb4a266224e7ca20ae34c9196b685)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:27.345982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-26ba19ab-823e-4925-965c-8cdfdc7bfc33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3727e43927674c2cb5b6f33ecd464ccd'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.346637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-26ba19ab-823e-4925-965c-8cdfdc7bfc33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3727e43927674c2cb5b6f33ecd464ccd'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.347346 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d20845-1148-4396-9bb5-cbf69c99f631 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:27.347900 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:27.356336 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-26ba19ab-823e-4925-965c-8cdfdc7bfc33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3727e43927674c2cb5b6f33ecd464ccd'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.369918 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-26ba19ab-823e-4925-965c-8cdfdc7bfc33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3727e43927674c2cb5b6f33ecd464ccd'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.379369 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-26ba19ab-823e-4925-965c-8cdfdc7bfc33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3727e43927674c2cb5b6f33ecd464ccd'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.386144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 939/1864] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:09:27 2022] DELETE /identity/v3/users/3727e43927674c2cb5b6f33ecd464ccd => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:27.403662 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-69d20845-1148-4396-9bb5-cbf69c99f631 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '290eb4a266224e7ca20ae34c9196b685', 'request_id': 'req-69d20845-1148-4396-9bb5-cbf69c99f631'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.411907 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-69d20845-1148-4396-9bb5-cbf69c99f631 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '290eb4a266224e7ca20ae34c9196b685', 'request_id': 'req-69d20845-1148-4396-9bb5-cbf69c99f631'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.423534 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-69d20845-1148-4396-9bb5-cbf69c99f631 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '290eb4a266224e7ca20ae34c9196b685', 'request_id': 'req-69d20845-1148-4396-9bb5-cbf69c99f631'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.437703 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-69d20845-1148-4396-9bb5-cbf69c99f631 admin admin] The token cache is being invalidate because project 290eb4a266224e7ca20ae34c9196b685 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:09:27.438097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-69d20845-1148-4396-9bb5-cbf69c99f631 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.439030 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-69d20845-1148-4396-9bb5-cbf69c99f631 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.441338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 926/1865] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:09:27 2022] DELETE /identity/v3/projects/290eb4a266224e7ca20ae34c9196b685 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:27.446026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba19ab-823e-4925-965c-8cdfdc7bfc33 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:27.447760 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-26ba19ab-823e-4925-965c-8cdfdc7bfc33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:27.519865 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba19ab-823e-4925-965c-8cdfdc7bfc33 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:27.524608 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37c0d1-f90d-4547-809a-ef99c8411052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:27.525246 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37c0d1-f90d-4547-809a-ef99c8411052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:27.525529 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37c0d1-f90d-4547-809a-ef99c8411052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:27.525803 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37c0d1-f90d-4547-809a-ef99c8411052 admin admin] PATH_INFO: `/v3/users/4e0f096d7a724d079869c0587b2adf55` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:27.526972 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37c0d1-f90d-4547-809a-ef99c8411052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e0f096d7a724d079869c0587b2adf55)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:27.528919 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37c0d1-f90d-4547-809a-ef99c8411052 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:27.582319 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-3f37c0d1-f90d-4547-809a-ef99c8411052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e0f096d7a724d079869c0587b2adf55'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.583003 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-3f37c0d1-f90d-4547-809a-ef99c8411052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e0f096d7a724d079869c0587b2adf55'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.590223 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-3f37c0d1-f90d-4547-809a-ef99c8411052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e0f096d7a724d079869c0587b2adf55'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.603202 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-3f37c0d1-f90d-4547-809a-ef99c8411052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e0f096d7a724d079869c0587b2adf55'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.611602 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-3f37c0d1-f90d-4547-809a-ef99c8411052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e0f096d7a724d079869c0587b2adf55'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.618790 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 940/1866] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:09:27 2022] DELETE /identity/v3/users/4e0f096d7a724d079869c0587b2adf55 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:09:27.705585 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d20845-1148-4396-9bb5-cbf69c99f631 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:27.707453 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-69d20845-1148-4396-9bb5-cbf69c99f631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:27.790927 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d20845-1148-4396-9bb5-cbf69c99f631 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:27.849910 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b5947e-413f-4f48-b858-06951644f6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:27.850538 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5947e-413f-4f48-b858-06951644f6a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:27.850821 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5947e-413f-4f48-b858-06951644f6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:27.851095 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5947e-413f-4f48-b858-06951644f6a9 admin admin] PATH_INFO: `/v3/projects/3f51ac081e72406ca83e055f039136d2` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:27.852203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b5947e-413f-4f48-b858-06951644f6a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f51ac081e72406ca83e055f039136d2)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:27.854194 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b5947e-413f-4f48-b858-06951644f6a9 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:27.863029 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:27.906193 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-60b5947e-413f-4f48-b858-06951644f6a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f51ac081e72406ca83e055f039136d2', 'request_id': 'req-60b5947e-413f-4f48-b858-06951644f6a9'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.912780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-60b5947e-413f-4f48-b858-06951644f6a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f51ac081e72406ca83e055f039136d2', 'request_id': 'req-60b5947e-413f-4f48-b858-06951644f6a9'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.915421 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37c0d1-f90d-4547-809a-ef99c8411052 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:27.916966 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-3f37c0d1-f90d-4547-809a-ef99c8411052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:27.922905 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-60b5947e-413f-4f48-b858-06951644f6a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f51ac081e72406ca83e055f039136d2', 'request_id': 'req-60b5947e-413f-4f48-b858-06951644f6a9'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.933763 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37c0d1-f90d-4547-809a-ef99c8411052 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:27.935270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-60b5947e-413f-4f48-b858-06951644f6a9 admin admin] The token cache is being invalidate because project 3f51ac081e72406ca83e055f039136d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:09:27.935639 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-60b5947e-413f-4f48-b858-06951644f6a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.936363 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-60b5947e-413f-4f48-b858-06951644f6a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:27.938002 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 927/1867] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:09:27 2022] DELETE /identity/v3/projects/3f51ac081e72406ca83e055f039136d2 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:27.942921 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358377e8-1341-4cad-9ff9-ef03f1a50f95 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:27.943276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358377e8-1341-4cad-9ff9-ef03f1a50f95 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:27.943464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358377e8-1341-4cad-9ff9-ef03f1a50f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:27.965024 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-358377e8-1341-4cad-9ff9-ef03f1a50f95 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:27.966255 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-358377e8-1341-4cad-9ff9-ef03f1a50f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:27.986021 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:28.003772 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599ac99-8723-4d20-8eb3-8e50acc6d6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:28.004439 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599ac99-8723-4d20-8eb3-8e50acc6d6ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:28.004757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599ac99-8723-4d20-8eb3-8e50acc6d6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:28.005036 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599ac99-8723-4d20-8eb3-8e50acc6d6ff admin admin] PATH_INFO: `/v3/projects/3a983949349947df9fd050dd3f5f6d45` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:28.006167 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599ac99-8723-4d20-8eb3-8e50acc6d6ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a983949349947df9fd050dd3f5f6d45)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:28.008095 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599ac99-8723-4d20-8eb3-8e50acc6d6ff admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:28.027599 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 928/1868] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:09:28.036403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfaf91-0493-4b3b-8c96-3ffa91a6987f None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:28.036833 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfaf91-0493-4b3b-8c96-3ffa91a6987f None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:28.037137 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfaf91-0493-4b3b-8c96-3ffa91a6987f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:28.060597 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-0599ac99-8723-4d20-8eb3-8e50acc6d6ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a983949349947df9fd050dd3f5f6d45', 'request_id': 'req-0599ac99-8723-4d20-8eb3-8e50acc6d6ff'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:28.060744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-43cfaf91-0493-4b3b-8c96-3ffa91a6987f None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:28.062033 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-43cfaf91-0493-4b3b-8c96-3ffa91a6987f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:28.074100 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-0599ac99-8723-4d20-8eb3-8e50acc6d6ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a983949349947df9fd050dd3f5f6d45', 'request_id': 'req-0599ac99-8723-4d20-8eb3-8e50acc6d6ff'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:28.080818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-0599ac99-8723-4d20-8eb3-8e50acc6d6ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a983949349947df9fd050dd3f5f6d45', 'request_id': 'req-0599ac99-8723-4d20-8eb3-8e50acc6d6ff'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:28.093507 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-0599ac99-8723-4d20-8eb3-8e50acc6d6ff admin admin] The token cache is being invalidate because project 3a983949349947df9fd050dd3f5f6d45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:09:28.094420 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-0599ac99-8723-4d20-8eb3-8e50acc6d6ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:28.095092 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-0599ac99-8723-4d20-8eb3-8e50acc6d6ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:28.096684 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 941/1869] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:09:27 2022] DELETE /identity/v3/projects/3a983949349947df9fd050dd3f5f6d45 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:09:28.106063 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b622795-faf7-4f81-b496-594eb04a9812 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:28.106063 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b622795-faf7-4f81-b496-594eb04a9812 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:28.106063 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b622795-faf7-4f81-b496-594eb04a9812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:28.128531 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-9b622795-faf7-4f81-b496-594eb04a9812 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:28.129883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9b622795-faf7-4f81-b496-594eb04a9812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:28.132078 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 929/1870] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:28.136204 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cfaf91-0493-4b3b-8c96-3ffa91a6987f None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:28.144643 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 942/1871] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:28.145665 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cfaf91-0493-4b3b-8c96-3ffa91a6987f None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:28.149925 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8eefa-90cb-45e3-a505-7af06dc8dc86 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:28.150211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8eefa-90cb-45e3-a505-7af06dc8dc86 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:28.150489 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8eefa-90cb-45e3-a505-7af06dc8dc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:28.157258 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f848aa-95fc-4fc5-991d-9e908e0608a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:28.158170 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f848aa-95fc-4fc5-991d-9e908e0608a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:28.158657 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f848aa-95fc-4fc5-991d-9e908e0608a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:28.159119 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f848aa-95fc-4fc5-991d-9e908e0608a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:28.159957 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f848aa-95fc-4fc5-991d-9e908e0608a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:09:28.160661 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f848aa-95fc-4fc5-991d-9e908e0608a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:28.170482 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f848aa-95fc-4fc5-991d-9e908e0608a8 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:28.182820 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-12d8eefa-90cb-45e3-a505-7af06dc8dc86 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:28.187501 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-12d8eefa-90cb-45e3-a505-7af06dc8dc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:28.193984 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 930/1872] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:09:28.195414 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f848aa-95fc-4fc5-991d-9e908e0608a8 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:28.204390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 943/1873] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:28.206201 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8eefa-90cb-45e3-a505-7af06dc8dc86 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:28.212100 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f848aa-95fc-4fc5-991d-9e908e0608a8 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:28.217243 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8eefa-90cb-45e3-a505-7af06dc8dc86 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:28.222387 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b3f0a-5a20-4f80-83e2-2c5d126d94d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:28.223124 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b3f0a-5a20-4f80-83e2-2c5d126d94d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:28.223483 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b3f0a-5a20-4f80-83e2-2c5d126d94d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:28.223898 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b3f0a-5a20-4f80-83e2-2c5d126d94d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:28.224006 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044122a-905d-45ad-aee9-15cd5e7ef0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:28.224708 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b3f0a-5a20-4f80-83e2-2c5d126d94d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:28.224988 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044122a-905d-45ad-aee9-15cd5e7ef0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:28.225281 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044122a-905d-45ad-aee9-15cd5e7ef0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:28.225707 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044122a-905d-45ad-aee9-15cd5e7ef0e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:28.226363 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9044122a-905d-45ad-aee9-15cd5e7ef0e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:09:28.226666 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9044122a-905d-45ad-aee9-15cd5e7ef0e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:28.227165 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17b3f0a-5a20-4f80-83e2-2c5d126d94d5 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:28.228242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9044122a-905d-45ad-aee9-15cd5e7ef0e7 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:28.248530 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 944/1874] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:09:28.252051 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044122a-905d-45ad-aee9-15cd5e7ef0e7 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:28.261583 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 931/1875] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:09:28 2022] POST /identity/v3/projects => generated 359 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:28.262860 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044122a-905d-45ad-aee9-15cd5e7ef0e7 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:28.265898 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b3f0a-5a20-4f80-83e2-2c5d126d94d5 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:28.270136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:28.322313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7c5d9-7557-43c6-bd8a-c35315561660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:28.322923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7c5d9-7557-43c6-bd8a-c35315561660 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:28.323209 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7c5d9-7557-43c6-bd8a-c35315561660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:28.323487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7c5d9-7557-43c6-bd8a-c35315561660 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:28.324117 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c7c5d9-7557-43c6-bd8a-c35315561660 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:28.326254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c7c5d9-7557-43c6-bd8a-c35315561660 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:28.331586 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b3f0a-5a20-4f80-83e2-2c5d126d94d5 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:28.336283 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c77e08-1d5a-457d-a836-97058d5160fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:28.336937 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c77e08-1d5a-457d-a836-97058d5160fa admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:28.337196 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c77e08-1d5a-457d-a836-97058d5160fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:28.337474 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c77e08-1d5a-457d-a836-97058d5160fa admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:28.338136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c77e08-1d5a-457d-a836-97058d5160fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:28.340020 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c77e08-1d5a-457d-a836-97058d5160fa admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:28.354899 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 945/1876] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:09:28 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:28.359759 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7c5d9-7557-43c6-bd8a-c35315561660 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:28.366330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 932/1877] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:09:28 2022] POST /identity/v3/users => generated 280 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:09:28.370074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c77e08-1d5a-457d-a836-97058d5160fa admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:28.379961 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c77e08-1d5a-457d-a836-97058d5160fa admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:28.394739 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:28.424742 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7c5d9-7557-43c6-bd8a-c35315561660 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:28.429474 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db508cc3-7f64-4e12-8de8-a49b3b87fae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:28.430084 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db508cc3-7f64-4e12-8de8-a49b3b87fae1 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:28.430365 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db508cc3-7f64-4e12-8de8-a49b3b87fae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:28.430642 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db508cc3-7f64-4e12-8de8-a49b3b87fae1 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:28.431280 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db508cc3-7f64-4e12-8de8-a49b3b87fae1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:28.433223 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db508cc3-7f64-4e12-8de8-a49b3b87fae1 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:28.440660 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219409b-523d-4b53-971e-99d0f9186728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:28.441598 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219409b-523d-4b53-971e-99d0f9186728 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:28.442036 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219409b-523d-4b53-971e-99d0f9186728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:28.442324 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219409b-523d-4b53-971e-99d0f9186728 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:28.442979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219409b-523d-4b53-971e-99d0f9186728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:28.444927 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219409b-523d-4b53-971e-99d0f9186728 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:28.459346 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 933/1878] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:09:28.459542 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 946/1879] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:09:28 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:09:28.462217 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219409b-523d-4b53-971e-99d0f9186728 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:28.462339 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db508cc3-7f64-4e12-8de8-a49b3b87fae1 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:28.473072 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db508cc3-7f64-4e12-8de8-a49b3b87fae1 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:28.473581 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219409b-523d-4b53-971e-99d0f9186728 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:28.477861 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33cb3a1-3c4d-4e3a-a559-909750e5f8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:28.478561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cb3a1-3c4d-4e3a-a559-909750e5f8ea admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:28.478924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cb3a1-3c4d-4e3a-a559-909750e5f8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:28.479374 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33cb3a1-3c4d-4e3a-a559-909750e5f8ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:28.480007 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72c035-22c4-4a1b-a2e6-6c00579c4c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:28.480230 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33cb3a1-3c4d-4e3a-a559-909750e5f8ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:28.480646 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72c035-22c4-4a1b-a2e6-6c00579c4c8b admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:28.480942 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72c035-22c4-4a1b-a2e6-6c00579c4c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:28.481219 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72c035-22c4-4a1b-a2e6-6c00579c4c8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:28.481858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad72c035-22c4-4a1b-a2e6-6c00579c4c8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:28.481972 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33cb3a1-3c4d-4e3a-a559-909750e5f8ea admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:28.483717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad72c035-22c4-4a1b-a2e6-6c00579c4c8b admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:28.494432 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 947/1880] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:09:28.505901 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 934/1881] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:28.506058 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33cb3a1-3c4d-4e3a-a559-909750e5f8ea admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:28.507281 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72c035-22c4-4a1b-a2e6-6c00579c4c8b admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:28.521173 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72c035-22c4-4a1b-a2e6-6c00579c4c8b admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:28.521432 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33cb3a1-3c4d-4e3a-a559-909750e5f8ea admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:28.525899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047cb751-2fcd-4644-a030-72b9212f89e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:28.526390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fb2fa-c06d-438f-8ace-40376a054e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:28.526619 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cb751-2fcd-4644-a030-72b9212f89e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:28.527193 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fb2fa-c06d-438f-8ace-40376a054e5e admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:28.527479 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cb751-2fcd-4644-a030-72b9212f89e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:28.527681 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fb2fa-c06d-438f-8ace-40376a054e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:28.527790 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cb751-2fcd-4644-a030-72b9212f89e1 admin admin] PATH_INFO: `/v3/projects/8022c412c1bb4415a19b6aaa0cdc0c8a/users/cbe5adb2a885467b828efc12d3e81444/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:28.527967 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fb2fa-c06d-438f-8ace-40376a054e5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:28.528654 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759fb2fa-c06d-438f-8ace-40376a054e5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:28.530241 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759fb2fa-c06d-438f-8ace-40376a054e5e admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:28.552963 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 948/1882] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:28.557151 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fb2fa-c06d-438f-8ace-40376a054e5e admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:28.557493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047cb751-2fcd-4644-a030-72b9212f89e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8022c412c1bb4415a19b6aaa0cdc0c8a, user_id=cbe5adb2a885467b828efc12d3e81444, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:28.560133 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047cb751-2fcd-4644-a030-72b9212f89e1 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:28.568172 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fb2fa-c06d-438f-8ace-40376a054e5e admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:28.573093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ad120-5dcb-4bf4-95cd-4167a77d6984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:28.573849 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ad120-5dcb-4bf4-95cd-4167a77d6984 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:28.574192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ad120-5dcb-4bf4-95cd-4167a77d6984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:28.574537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ad120-5dcb-4bf4-95cd-4167a77d6984 admin admin] PATH_INFO: `/v3/projects/9398746f87a54e39999c7e4d35e60bad/users/0c75ef8131ab47ec860b777df25f6e67/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:28.575093 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 935/1883] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:28 2022] PUT /identity/v3/projects/8022c412c1bb4415a19b6aaa0cdc0c8a/users/cbe5adb2a885467b828efc12d3e81444/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:09:28.586353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca69ce8-7602-4908-9034-2c98508ecd21 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:28.586722 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca69ce8-7602-4908-9034-2c98508ecd21 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:28.587075 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca69ce8-7602-4908-9034-2c98508ecd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:28.593606 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ad120-5dcb-4bf4-95cd-4167a77d6984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9398746f87a54e39999c7e4d35e60bad, user_id=0c75ef8131ab47ec860b777df25f6e67, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:28.596511 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ad120-5dcb-4bf4-95cd-4167a77d6984 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:28.610217 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 949/1884] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:28 2022] PUT /identity/v3/projects/9398746f87a54e39999c7e4d35e60bad/users/0c75ef8131ab47ec860b777df25f6e67/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:09:28.616260 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-bca69ce8-7602-4908-9034-2c98508ecd21 None None] MFA Rules not processed for user `cbe5adb2a885467b828efc12d3e81444`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:28.617494 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-bca69ce8-7602-4908-9034-2c98508ecd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:28.620254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be74c89-ace9-4e83-b441-426282fac575 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:28.620586 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be74c89-ace9-4e83-b441-426282fac575 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:28.620873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be74c89-ace9-4e83-b441-426282fac575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:28.648776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-8be74c89-ace9-4e83-b441-426282fac575 None None] MFA Rules not processed for user `0c75ef8131ab47ec860b777df25f6e67`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:28.649993 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8be74c89-ace9-4e83-b441-426282fac575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:28.663005 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:28.687916 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 936/1885] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:28 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:09:28.720517 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 950/1886] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:09:28.725753 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca69ce8-7602-4908-9034-2c98508ecd21 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:28.727243 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-bca69ce8-7602-4908-9034-2c98508ecd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:28.727413 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be74c89-ace9-4e83-b441-426282fac575 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:28.728767 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8be74c89-ace9-4e83-b441-426282fac575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:28.789154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be74c89-ace9-4e83-b441-426282fac575 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:28.789578 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca69ce8-7602-4908-9034-2c98508ecd21 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:28.792884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585e0c0-d6a2-4e39-9080-6c3070aec43e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:28.793225 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad69ad5-66d6-462a-a365-ec63afae757b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:28.793496 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585e0c0-d6a2-4e39-9080-6c3070aec43e service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:28.793781 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585e0c0-d6a2-4e39-9080-6c3070aec43e service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:28.793880 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad69ad5-66d6-462a-a365-ec63afae757b service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:28.794060 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585e0c0-d6a2-4e39-9080-6c3070aec43e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:28.794166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad69ad5-66d6-462a-a365-ec63afae757b service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:28.794414 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad69ad5-66d6-462a-a365-ec63afae757b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:28.803195 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585e0c0-d6a2-4e39-9080-6c3070aec43e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:28.804470 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad69ad5-66d6-462a-a365-ec63afae757b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:28.804654 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585e0c0-d6a2-4e39-9080-6c3070aec43e service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:28.805914 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad69ad5-66d6-462a-a365-ec63afae757b service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:28.814204 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 951/1887] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:09:28.815083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 937/1888] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:28 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:09:28.832509 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585e0c0-d6a2-4e39-9080-6c3070aec43e service cinder] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:28.849225 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:28.900967 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585e0c0-d6a2-4e39-9080-6c3070aec43e service cinder] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:28.906636 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a46b05-9bde-43ff-8895-7a0728fa8a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:28.907267 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a46b05-9bde-43ff-8895-7a0728fa8a69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:28.907548 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a46b05-9bde-43ff-8895-7a0728fa8a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:28.907826 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a46b05-9bde-43ff-8895-7a0728fa8a69 admin admin] PATH_INFO: `/v3/users/cbe5adb2a885467b828efc12d3e81444` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:28.909001 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a46b05-9bde-43ff-8895-7a0728fa8a69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbe5adb2a885467b828efc12d3e81444)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:28.910940 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a46b05-9bde-43ff-8895-7a0728fa8a69 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:28.959519 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-24a46b05-9bde-43ff-8895-7a0728fa8a69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbe5adb2a885467b828efc12d3e81444'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:28.960373 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-24a46b05-9bde-43ff-8895-7a0728fa8a69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbe5adb2a885467b828efc12d3e81444'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:28.968286 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-24a46b05-9bde-43ff-8895-7a0728fa8a69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbe5adb2a885467b828efc12d3e81444'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:28.981569 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-24a46b05-9bde-43ff-8895-7a0728fa8a69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbe5adb2a885467b828efc12d3e81444'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:28.990424 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-24a46b05-9bde-43ff-8895-7a0728fa8a69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbe5adb2a885467b828efc12d3e81444'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:28.996722 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 952/1889] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:09:28 2022] DELETE /identity/v3/users/cbe5adb2a885467b828efc12d3e81444 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:29.001304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2552409-33f8-448a-9d5e-eb361f1f1e76 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:29.001685 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2552409-33f8-448a-9d5e-eb361f1f1e76 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:29.001909 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2552409-33f8-448a-9d5e-eb361f1f1e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:29.021140 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-c2552409-33f8-448a-9d5e-eb361f1f1e76 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:29.022349 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-c2552409-33f8-448a-9d5e-eb361f1f1e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:29.089700 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 938/1890] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:09:29.106707 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a46b05-9bde-43ff-8895-7a0728fa8a69 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:29.106707 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-24a46b05-9bde-43ff-8895-7a0728fa8a69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:29.172826 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a46b05-9bde-43ff-8895-7a0728fa8a69 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:29.176457 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e45873-6056-4861-8306-759c3e369525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:29.177102 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e45873-6056-4861-8306-759c3e369525 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:29.177388 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e45873-6056-4861-8306-759c3e369525 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:29.177665 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e45873-6056-4861-8306-759c3e369525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:29.187327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e45873-6056-4861-8306-759c3e369525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:29.188794 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e45873-6056-4861-8306-759c3e369525 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:29.198104 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 953/1891] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:09:29.473920 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2552409-33f8-448a-9d5e-eb361f1f1e76 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:29.475469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-c2552409-33f8-448a-9d5e-eb361f1f1e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:29.493461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2552409-33f8-448a-9d5e-eb361f1f1e76 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:29.498370 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e67be-4e5f-4a45-b1df-f40d88f9dea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:29.498927 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e67be-4e5f-4a45-b1df-f40d88f9dea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:29.499709 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e67be-4e5f-4a45-b1df-f40d88f9dea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:29.500400 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e67be-4e5f-4a45-b1df-f40d88f9dea9 admin admin] PATH_INFO: `/v3/projects/8022c412c1bb4415a19b6aaa0cdc0c8a` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:29.501521 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e67be-4e5f-4a45-b1df-f40d88f9dea9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8022c412c1bb4415a19b6aaa0cdc0c8a)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:29.503414 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e67be-4e5f-4a45-b1df-f40d88f9dea9 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:29.553610 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-073e67be-4e5f-4a45-b1df-f40d88f9dea9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8022c412c1bb4415a19b6aaa0cdc0c8a', 'request_id': 'req-073e67be-4e5f-4a45-b1df-f40d88f9dea9'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:29.569793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-073e67be-4e5f-4a45-b1df-f40d88f9dea9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8022c412c1bb4415a19b6aaa0cdc0c8a', 'request_id': 'req-073e67be-4e5f-4a45-b1df-f40d88f9dea9'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:29.579616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-073e67be-4e5f-4a45-b1df-f40d88f9dea9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8022c412c1bb4415a19b6aaa0cdc0c8a', 'request_id': 'req-073e67be-4e5f-4a45-b1df-f40d88f9dea9'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:29.590565 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-073e67be-4e5f-4a45-b1df-f40d88f9dea9 admin admin] The token cache is being invalidate because project 8022c412c1bb4415a19b6aaa0cdc0c8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:09:29.590979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-073e67be-4e5f-4a45-b1df-f40d88f9dea9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:29.591694 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-073e67be-4e5f-4a45-b1df-f40d88f9dea9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:29.594315 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 939/1892] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:09:29 2022] DELETE /identity/v3/projects/8022c412c1bb4415a19b6aaa0cdc0c8a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:29.599117 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e32c9-7273-4c2a-bae4-a5abbb7ba739 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:29.599564 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e32c9-7273-4c2a-bae4-a5abbb7ba739 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:29.599670 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e32c9-7273-4c2a-bae4-a5abbb7ba739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:29.619390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-298e32c9-7273-4c2a-bae4-a5abbb7ba739 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:29.620667 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-298e32c9-7273-4c2a-bae4-a5abbb7ba739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:29.647049 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:29.682390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 954/1893] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:29.690701 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45b75c-d29c-4a38-a027-5cf76732c22b None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:29.690986 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45b75c-d29c-4a38-a027-5cf76732c22b None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:29.691264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45b75c-d29c-4a38-a027-5cf76732c22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:29.711012 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-8a45b75c-d29c-4a38-a027-5cf76732c22b None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:29.712194 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-8a45b75c-d29c-4a38-a027-5cf76732c22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:29.721241 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 940/1894] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:29.725499 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e32c9-7273-4c2a-bae4-a5abbb7ba739 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:29.735262 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e32c9-7273-4c2a-bae4-a5abbb7ba739 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:29.739817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849b20e-dd9e-4b96-adc9-8f18aa51bbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:29.740402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849b20e-dd9e-4b96-adc9-8f18aa51bbb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:29.740712 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849b20e-dd9e-4b96-adc9-8f18aa51bbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:29.740991 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849b20e-dd9e-4b96-adc9-8f18aa51bbb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:29.741625 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9849b20e-dd9e-4b96-adc9-8f18aa51bbb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:09:29.741928 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9849b20e-dd9e-4b96-adc9-8f18aa51bbb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:29.743498 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9849b20e-dd9e-4b96-adc9-8f18aa51bbb7 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:29.755791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 955/1895] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:09:29.760130 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45b75c-d29c-4a38-a027-5cf76732c22b None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:29.764547 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:29.771500 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45b75c-d29c-4a38-a027-5cf76732c22b None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:29.777170 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b103b9-165b-439d-a6b7-f66c06bacc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:29.777773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b103b9-165b-439d-a6b7-f66c06bacc26 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:29.778057 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b103b9-165b-439d-a6b7-f66c06bacc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:29.778338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b103b9-165b-439d-a6b7-f66c06bacc26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:29.778977 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b103b9-165b-439d-a6b7-f66c06bacc26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:29.781647 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b103b9-165b-439d-a6b7-f66c06bacc26 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:29.810148 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 941/1896] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:09:29 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:29.813353 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849b20e-dd9e-4b96-adc9-8f18aa51bbb7 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:29.823509 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849b20e-dd9e-4b96-adc9-8f18aa51bbb7 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:29.882790 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedac303-e3f1-40d3-a4cb-89e756277e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:29.883427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedac303-e3f1-40d3-a4cb-89e756277e0b admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:29.883739 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedac303-e3f1-40d3-a4cb-89e756277e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:29.884016 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedac303-e3f1-40d3-a4cb-89e756277e0b admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:29.884686 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedac303-e3f1-40d3-a4cb-89e756277e0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:29.886583 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedac303-e3f1-40d3-a4cb-89e756277e0b admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:29.907607 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 956/1897] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:09:29 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:29.911857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b103b9-165b-439d-a6b7-f66c06bacc26 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:29.922999 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b103b9-165b-439d-a6b7-f66c06bacc26 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:29.927578 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e760441-28f3-4e06-b3b9-d445bd5f14c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:29.928170 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e760441-28f3-4e06-b3b9-d445bd5f14c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:29.928454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e760441-28f3-4e06-b3b9-d445bd5f14c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:29.928766 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e760441-28f3-4e06-b3b9-d445bd5f14c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:29.929489 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e760441-28f3-4e06-b3b9-d445bd5f14c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:29.931361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e760441-28f3-4e06-b3b9-d445bd5f14c9 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:29.942373 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 942/1898] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:29.946361 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedac303-e3f1-40d3-a4cb-89e756277e0b admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:29.956825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedac303-e3f1-40d3-a4cb-89e756277e0b admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:29.961468 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132c7d0-127f-4ed2-ae7f-296e56871ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:29.962062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132c7d0-127f-4ed2-ae7f-296e56871ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:29.962346 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132c7d0-127f-4ed2-ae7f-296e56871ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:29.962623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132c7d0-127f-4ed2-ae7f-296e56871ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:29.963257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132c7d0-127f-4ed2-ae7f-296e56871ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:29.964887 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132c7d0-127f-4ed2-ae7f-296e56871ff0 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:29.976328 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 957/1899] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:29.980410 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e760441-28f3-4e06-b3b9-d445bd5f14c9 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:29.990603 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e760441-28f3-4e06-b3b9-d445bd5f14c9 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:29.995243 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa1572-dea4-48e0-8a26-83c7f29fc168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:29.995885 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa1572-dea4-48e0-8a26-83c7f29fc168 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:29.996170 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa1572-dea4-48e0-8a26-83c7f29fc168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:29.996447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa1572-dea4-48e0-8a26-83c7f29fc168 admin admin] PATH_INFO: `/v3/projects/26754ba666844ff39d7d0e3a67f95d5f/users/18c1e97fc35343e796b9d1122cb1159b/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:30.010929 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa1572-dea4-48e0-8a26-83c7f29fc168 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26754ba666844ff39d7d0e3a67f95d5f, user_id=18c1e97fc35343e796b9d1122cb1159b, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:30.013627 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa1572-dea4-48e0-8a26-83c7f29fc168 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:30.027121 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 943/1900] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:29 2022] PUT /identity/v3/projects/26754ba666844ff39d7d0e3a67f95d5f/users/18c1e97fc35343e796b9d1122cb1159b/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:09:30.038136 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f41e4-c82d-4f7a-a4bc-7f5f77755517 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:30.038430 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f41e4-c82d-4f7a-a4bc-7f5f77755517 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:30.038737 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f41e4-c82d-4f7a-a4bc-7f5f77755517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:30.067634 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-c68f41e4-c82d-4f7a-a4bc-7f5f77755517 None None] MFA Rules not processed for user `18c1e97fc35343e796b9d1122cb1159b`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:30.068890 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c68f41e4-c82d-4f7a-a4bc-7f5f77755517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:30.135836 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 958/1901] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:09:30.147730 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa1572-dea4-48e0-8a26-83c7f29fc168 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:30.149490 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-c9fa1572-dea4-48e0-8a26-83c7f29fc168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:30.209453 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa1572-dea4-48e0-8a26-83c7f29fc168 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:30.213238 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca2ccf-5d08-4aea-a0c9-b6790ddbeb52 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a69aeb3fba49b6881f1f15c1758494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:30.213884 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca2ccf-5d08-4aea-a0c9-b6790ddbeb52 service glance] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:30.214215 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca2ccf-5d08-4aea-a0c9-b6790ddbeb52 service glance] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:30.214499 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca2ccf-5d08-4aea-a0c9-b6790ddbeb52 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:30.223932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca2ccf-5d08-4aea-a0c9-b6790ddbeb52 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:30.225486 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca2ccf-5d08-4aea-a0c9-b6790ddbeb52 service glance] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:30.236450 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 944/1902] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:09:31.570717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f41e4-c82d-4f7a-a4bc-7f5f77755517 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:31.572142 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c68f41e4-c82d-4f7a-a4bc-7f5f77755517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:31.602657 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:31.632588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f41e4-c82d-4f7a-a4bc-7f5f77755517 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:31.637080 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bef71-9358-47d9-891c-df7e87e4e754 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:31.637715 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bef71-9358-47d9-891c-df7e87e4e754 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:31.638002 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bef71-9358-47d9-891c-df7e87e4e754 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:31.638278 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bef71-9358-47d9-891c-df7e87e4e754 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:31.640799 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d34bef71-9358-47d9-891c-df7e87e4e754 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:31.708801 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34bef71-9358-47d9-891c-df7e87e4e754 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:31.710378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34bef71-9358-47d9-891c-df7e87e4e754 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:31.720909 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 959/1903] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:09:34.280877 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca2ccf-5d08-4aea-a0c9-b6790ddbeb52 service glance] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:34.282531 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-83ca2ccf-5d08-4aea-a0c9-b6790ddbeb52 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:34.294145 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca2ccf-5d08-4aea-a0c9-b6790ddbeb52 service glance] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:34.298428 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f841e15-3e70-4faf-950f-bf8979f40a81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:34.299024 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f841e15-3e70-4faf-950f-bf8979f40a81 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:34.299305 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f841e15-3e70-4faf-950f-bf8979f40a81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:34.299580 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f841e15-3e70-4faf-950f-bf8979f40a81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:34.301271 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-4f841e15-3e70-4faf-950f-bf8979f40a81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:34.366810 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f841e15-3e70-4faf-950f-bf8979f40a81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:34.368266 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f841e15-3e70-4faf-950f-bf8979f40a81 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:34.378132 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 945/1904] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:09:39.057212 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bef71-9358-47d9-891c-df7e87e4e754 service cinder] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:39.058689 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d34bef71-9358-47d9-891c-df7e87e4e754 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:39.125604 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bef71-9358-47d9-891c-df7e87e4e754 service cinder] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:39.130156 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb122391-c97d-45a6-b371-0a380ada69a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:39.130782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb122391-c97d-45a6-b371-0a380ada69a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:39.131061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb122391-c97d-45a6-b371-0a380ada69a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:39.131340 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb122391-c97d-45a6-b371-0a380ada69a3 admin admin] PATH_INFO: `/v3/users/e343e4b895854d2e852afae3953de172` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:39.132448 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb122391-c97d-45a6-b371-0a380ada69a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e343e4b895854d2e852afae3953de172)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:39.134362 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb122391-c97d-45a6-b371-0a380ada69a3 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:39.182416 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fb122391-c97d-45a6-b371-0a380ada69a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e343e4b895854d2e852afae3953de172'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:39.183069 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fb122391-c97d-45a6-b371-0a380ada69a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e343e4b895854d2e852afae3953de172'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:39.191093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fb122391-c97d-45a6-b371-0a380ada69a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e343e4b895854d2e852afae3953de172'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:39.203923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fb122391-c97d-45a6-b371-0a380ada69a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e343e4b895854d2e852afae3953de172'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:39.212764 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fb122391-c97d-45a6-b371-0a380ada69a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e343e4b895854d2e852afae3953de172'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:39.215874 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:39.228850 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 960/1905] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:09:39 2022] DELETE /identity/v3/users/e343e4b895854d2e852afae3953de172 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:39.233453 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903e632-e07b-49dd-90b0-cc747811b38f None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:39.234620 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903e632-e07b-49dd-90b0-cc747811b38f None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:39.234620 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903e632-e07b-49dd-90b0-cc747811b38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:39.254533 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-5903e632-e07b-49dd-90b0-cc747811b38f None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:39.255843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5903e632-e07b-49dd-90b0-cc747811b38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:39.306311 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:39.333275 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 946/1906] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:39.345282 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb122391-c97d-45a6-b371-0a380ada69a3 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:39.346693 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-fb122391-c97d-45a6-b371-0a380ada69a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:39.423208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb122391-c97d-45a6-b371-0a380ada69a3 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:39.427171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73d9c1-94b3-4966-98c2-d48863c9a49a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:39.427729 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73d9c1-94b3-4966-98c2-d48863c9a49a service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:39.428021 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73d9c1-94b3-4966-98c2-d48863c9a49a service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:39.428301 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73d9c1-94b3-4966-98c2-d48863c9a49a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:39.439041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73d9c1-94b3-4966-98c2-d48863c9a49a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:39.440416 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73d9c1-94b3-4966-98c2-d48863c9a49a service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:39.453359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 961/1907] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:09:39.748833 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903e632-e07b-49dd-90b0-cc747811b38f None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:39.751570 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5903e632-e07b-49dd-90b0-cc747811b38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:39.776038 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903e632-e07b-49dd-90b0-cc747811b38f None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:39.781207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9d309-4967-433d-9885-5c93023139d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:39.781862 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9d309-4967-433d-9885-5c93023139d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:39.782045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9d309-4967-433d-9885-5c93023139d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:39.782324 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9d309-4967-433d-9885-5c93023139d1 admin admin] PATH_INFO: `/v3/projects/56497cbfe627422fb354905e112446c2` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:39.783452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9d309-4967-433d-9885-5c93023139d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56497cbfe627422fb354905e112446c2)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:39.785442 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9d309-4967-433d-9885-5c93023139d1 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:39.835374 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d6a9d309-4967-433d-9885-5c93023139d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56497cbfe627422fb354905e112446c2', 'request_id': 'req-d6a9d309-4967-433d-9885-5c93023139d1'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:39.843627 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d6a9d309-4967-433d-9885-5c93023139d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56497cbfe627422fb354905e112446c2', 'request_id': 'req-d6a9d309-4967-433d-9885-5c93023139d1'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:39.851436 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d6a9d309-4967-433d-9885-5c93023139d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56497cbfe627422fb354905e112446c2', 'request_id': 'req-d6a9d309-4967-433d-9885-5c93023139d1'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:39.862842 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d6a9d309-4967-433d-9885-5c93023139d1 admin admin] The token cache is being invalidate because project 56497cbfe627422fb354905e112446c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:09:39.863132 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d6a9d309-4967-433d-9885-5c93023139d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:39.863743 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d6a9d309-4967-433d-9885-5c93023139d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:39.865251 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 947/1908] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:09:39 2022] DELETE /identity/v3/projects/56497cbfe627422fb354905e112446c2 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:39.870149 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee649ff3-ad85-4dac-b866-39fd91af4b28 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:39.870488 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee649ff3-ad85-4dac-b866-39fd91af4b28 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:39.870773 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee649ff3-ad85-4dac-b866-39fd91af4b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:39.891491 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-ee649ff3-ad85-4dac-b866-39fd91af4b28 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:39.892715 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ee649ff3-ad85-4dac-b866-39fd91af4b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:39.958434 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 962/1909] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:09:39.972259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a078744-0456-47ae-9197-f91cb340e5eb None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:39.972616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a078744-0456-47ae-9197-f91cb340e5eb None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:39.972894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a078744-0456-47ae-9197-f91cb340e5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:39.995007 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-9a078744-0456-47ae-9197-f91cb340e5eb None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:39.996307 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9a078744-0456-47ae-9197-f91cb340e5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:40.004348 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 948/1910] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:40.008554 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee649ff3-ad85-4dac-b866-39fd91af4b28 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:40.018426 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee649ff3-ad85-4dac-b866-39fd91af4b28 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:40.023372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03656193-1624-469a-a5c1-8b29e45713dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:40.023987 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03656193-1624-469a-a5c1-8b29e45713dc admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:40.024265 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03656193-1624-469a-a5c1-8b29e45713dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:40.024575 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03656193-1624-469a-a5c1-8b29e45713dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:40.025225 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03656193-1624-469a-a5c1-8b29e45713dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:09:40.025529 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03656193-1624-469a-a5c1-8b29e45713dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:40.027197 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03656193-1624-469a-a5c1-8b29e45713dc admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:40.040348 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 963/1911] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:09:40.045588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a078744-0456-47ae-9197-f91cb340e5eb None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:40.055235 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a078744-0456-47ae-9197-f91cb340e5eb None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:40.060171 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028fcf43-9d83-40ad-ad96-dd4d995aa50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:40.061111 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028fcf43-9d83-40ad-ad96-dd4d995aa50f admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:40.061111 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028fcf43-9d83-40ad-ad96-dd4d995aa50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:40.061412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028fcf43-9d83-40ad-ad96-dd4d995aa50f admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:40.062713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028fcf43-9d83-40ad-ad96-dd4d995aa50f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:40.064689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028fcf43-9d83-40ad-ad96-dd4d995aa50f admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:40.093816 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 949/1912] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:09:40 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:40.098221 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03656193-1624-469a-a5c1-8b29e45713dc admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:40.113944 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03656193-1624-469a-a5c1-8b29e45713dc admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:40.173083 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993a1e5-c1b4-481b-9ae1-28f0d1e60687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:40.173712 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993a1e5-c1b4-481b-9ae1-28f0d1e60687 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:40.173999 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993a1e5-c1b4-481b-9ae1-28f0d1e60687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:40.174303 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993a1e5-c1b4-481b-9ae1-28f0d1e60687 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:40.175011 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993a1e5-c1b4-481b-9ae1-28f0d1e60687 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:40.177086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c993a1e5-c1b4-481b-9ae1-28f0d1e60687 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:40.198330 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 964/1913] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:09:40 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:09:40.202167 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028fcf43-9d83-40ad-ad96-dd4d995aa50f admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:40.212447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028fcf43-9d83-40ad-ad96-dd4d995aa50f admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:40.217003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3caac-4d62-43b3-86ee-9c2a04205228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:40.217595 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3caac-4d62-43b3-86ee-9c2a04205228 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:40.218401 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3caac-4d62-43b3-86ee-9c2a04205228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:40.218723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3caac-4d62-43b3-86ee-9c2a04205228 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:40.219608 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd3caac-4d62-43b3-86ee-9c2a04205228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:40.221208 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd3caac-4d62-43b3-86ee-9c2a04205228 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:40.231074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 950/1914] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:40.235715 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993a1e5-c1b4-481b-9ae1-28f0d1e60687 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:40.245869 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993a1e5-c1b4-481b-9ae1-28f0d1e60687 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:40.250690 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e8cf0-25e5-4aef-ad17-6ebf6692862c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:40.251296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e8cf0-25e5-4aef-ad17-6ebf6692862c admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:40.251585 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e8cf0-25e5-4aef-ad17-6ebf6692862c admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:40.251868 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e8cf0-25e5-4aef-ad17-6ebf6692862c admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:40.252559 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e8cf0-25e5-4aef-ad17-6ebf6692862c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:40.254114 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e8cf0-25e5-4aef-ad17-6ebf6692862c admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:40.264838 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 965/1915] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:09:40.269109 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3caac-4d62-43b3-86ee-9c2a04205228 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:40.279579 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3caac-4d62-43b3-86ee-9c2a04205228 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:40.285243 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3c880-7d19-46cc-94e5-d2084c1baa5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:40.285904 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3c880-7d19-46cc-94e5-d2084c1baa5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:40.286192 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3c880-7d19-46cc-94e5-d2084c1baa5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:40.286475 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a3c880-7d19-46cc-94e5-d2084c1baa5d admin admin] PATH_INFO: `/v3/projects/61c69232b9d04e1688d00c44f3da647e/users/d28f7a2f7130426da0d5a8d1ff943582/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:40.301852 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a3c880-7d19-46cc-94e5-d2084c1baa5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=61c69232b9d04e1688d00c44f3da647e, user_id=d28f7a2f7130426da0d5a8d1ff943582, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:40.304505 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a3c880-7d19-46cc-94e5-d2084c1baa5d admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:40.317422 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 951/1916] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:40 2022] PUT /identity/v3/projects/61c69232b9d04e1688d00c44f3da647e/users/d28f7a2f7130426da0d5a8d1ff943582/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:09:40.329505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bc6f4-ecfc-4c10-b528-b934ee38d200 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:40.329798 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bc6f4-ecfc-4c10-b528-b934ee38d200 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:40.330080 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bc6f4-ecfc-4c10-b528-b934ee38d200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:40.350981 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:40.361274 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-274bc6f4-ecfc-4c10-b528-b934ee38d200 None None] MFA Rules not processed for user `d28f7a2f7130426da0d5a8d1ff943582`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:40.362523 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-274bc6f4-ecfc-4c10-b528-b934ee38d200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:40.427306 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 966/1917] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:09:40.431797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3c880-7d19-46cc-94e5-d2084c1baa5d admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:40.502409 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a3c880-7d19-46cc-94e5-d2084c1baa5d admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:40.508379 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43407c8-68c8-4eb2-83d7-3cc70d9de8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:40.509026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43407c8-68c8-4eb2-83d7-3cc70d9de8ef admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:40.509314 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43407c8-68c8-4eb2-83d7-3cc70d9de8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:40.509599 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43407c8-68c8-4eb2-83d7-3cc70d9de8ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:40.510248 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43407c8-68c8-4eb2-83d7-3cc70d9de8ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:40.512145 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43407c8-68c8-4eb2-83d7-3cc70d9de8ef admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:40.517889 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:40.544700 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 952/1918] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:09:40 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:40.547986 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274bc6f4-ecfc-4c10-b528-b934ee38d200 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:40.558775 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274bc6f4-ecfc-4c10-b528-b934ee38d200 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:40.619044 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491be3a2-ca11-437b-a48c-c4657510c832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:40.619660 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491be3a2-ca11-437b-a48c-c4657510c832 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:40.619948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491be3a2-ca11-437b-a48c-c4657510c832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:40.620227 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491be3a2-ca11-437b-a48c-c4657510c832 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:40.620903 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491be3a2-ca11-437b-a48c-c4657510c832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:40.622819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491be3a2-ca11-437b-a48c-c4657510c832 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:40.644042 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 967/1919] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:09:40 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:40.648065 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43407c8-68c8-4eb2-83d7-3cc70d9de8ef admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:40.658527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43407c8-68c8-4eb2-83d7-3cc70d9de8ef admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:40.663279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438dd3e1-614e-47c4-9f02-2af6dfc24240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:40.663893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438dd3e1-614e-47c4-9f02-2af6dfc24240 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:40.664163 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438dd3e1-614e-47c4-9f02-2af6dfc24240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:40.664443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438dd3e1-614e-47c4-9f02-2af6dfc24240 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:40.665120 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438dd3e1-614e-47c4-9f02-2af6dfc24240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:40.666696 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438dd3e1-614e-47c4-9f02-2af6dfc24240 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:40.677250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 953/1920] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:40.681300 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491be3a2-ca11-437b-a48c-c4657510c832 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:40.691781 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491be3a2-ca11-437b-a48c-c4657510c832 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:40.696449 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734af3fd-5001-4edd-8fef-9e709b1eefe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:40.697139 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734af3fd-5001-4edd-8fef-9e709b1eefe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:40.697426 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734af3fd-5001-4edd-8fef-9e709b1eefe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:40.697709 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734af3fd-5001-4edd-8fef-9e709b1eefe6 admin admin] PATH_INFO: `/v3/projects/36d6eab1c640472391f0794ffae48a0d/users/3ed0722ac7dc4240b97f805fd67f1731/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:40.720677 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734af3fd-5001-4edd-8fef-9e709b1eefe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36d6eab1c640472391f0794ffae48a0d, user_id=3ed0722ac7dc4240b97f805fd67f1731, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:40.723484 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734af3fd-5001-4edd-8fef-9e709b1eefe6 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:40.736791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 968/1921] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:40 2022] PUT /identity/v3/projects/36d6eab1c640472391f0794ffae48a0d/users/3ed0722ac7dc4240b97f805fd67f1731/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:40.740786 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438dd3e1-614e-47c4-9f02-2af6dfc24240 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:40.751460 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438dd3e1-614e-47c4-9f02-2af6dfc24240 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:40.815233 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3f03e-a8a5-4039-9861-9d5b14254c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:40.815850 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3f03e-a8a5-4039-9861-9d5b14254c66 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:40.816133 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3f03e-a8a5-4039-9861-9d5b14254c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:40.816410 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3f03e-a8a5-4039-9861-9d5b14254c66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:40.817094 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3f03e-a8a5-4039-9861-9d5b14254c66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:40.818679 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3f03e-a8a5-4039-9861-9d5b14254c66 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:40.831418 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 954/1922] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:40.835576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734af3fd-5001-4edd-8fef-9e709b1eefe6 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:40.847080 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734af3fd-5001-4edd-8fef-9e709b1eefe6 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:40.851846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea3e98-88a0-47cb-9c4e-db9f6687895a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:40.852523 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea3e98-88a0-47cb-9c4e-db9f6687895a admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:40.852822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea3e98-88a0-47cb-9c4e-db9f6687895a admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:40.853100 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea3e98-88a0-47cb-9c4e-db9f6687895a admin admin] PATH_INFO: `/v3/projects/36d6eab1c640472391f0794ffae48a0d/users/3ed0722ac7dc4240b97f805fd67f1731/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:40.855021 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea3e98-88a0-47cb-9c4e-db9f6687895a admin admin] RBAC: Authorizing `identity:create_grant(project_id=36d6eab1c640472391f0794ffae48a0d, user_id=3ed0722ac7dc4240b97f805fd67f1731, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:40.857660 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea3e98-88a0-47cb-9c4e-db9f6687895a admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:40.868783 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 969/1923] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:40 2022] PUT /identity/v3/projects/36d6eab1c640472391f0794ffae48a0d/users/3ed0722ac7dc4240b97f805fd67f1731/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:40.879887 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2a05f-0226-4f3c-9681-71b9ca77243d None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:40.880252 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2a05f-0226-4f3c-9681-71b9ca77243d None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:40.880571 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2a05f-0226-4f3c-9681-71b9ca77243d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:40.908120 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-c0e2a05f-0226-4f3c-9681-71b9ca77243d None None] MFA Rules not processed for user `3ed0722ac7dc4240b97f805fd67f1731`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:40.909347 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-c0e2a05f-0226-4f3c-9681-71b9ca77243d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:40.977528 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 955/1924] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:09:40.989144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea3e98-88a0-47cb-9c4e-db9f6687895a admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:40.990892 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f6ea3e98-88a0-47cb-9c4e-db9f6687895a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:41.052684 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea3e98-88a0-47cb-9c4e-db9f6687895a admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:41.056270 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0abba-26c1-45eb-91ed-1632229bf9ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:41.056906 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0abba-26c1-45eb-91ed-1632229bf9ab service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:41.057192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0abba-26c1-45eb-91ed-1632229bf9ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:41.057465 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0abba-26c1-45eb-91ed-1632229bf9ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:41.061575 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:41.122245 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0abba-26c1-45eb-91ed-1632229bf9ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:41.124977 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0abba-26c1-45eb-91ed-1632229bf9ab service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:41.139629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 970/1925] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:09:50.974181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=73,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:51.132953 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=25 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:52.791438 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2a05f-0226-4f3c-9681-71b9ca77243d None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:52.792913 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-c0e2a05f-0226-4f3c-9681-71b9ca77243d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:52.862704 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2a05f-0226-4f3c-9681-71b9ca77243d None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:52.867253 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c48c68-094d-40ee-a818-1c120b134af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:52.867882 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c48c68-094d-40ee-a818-1c120b134af4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:52.868169 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c48c68-094d-40ee-a818-1c120b134af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:52.868453 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c48c68-094d-40ee-a818-1c120b134af4 admin admin] PATH_INFO: `/v3/users/0c75ef8131ab47ec860b777df25f6e67` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:52.869611 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c48c68-094d-40ee-a818-1c120b134af4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c75ef8131ab47ec860b777df25f6e67)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:52.871503 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c48c68-094d-40ee-a818-1c120b134af4 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:52.922911 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-e3c48c68-094d-40ee-a818-1c120b134af4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c75ef8131ab47ec860b777df25f6e67'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:52.930551 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-e3c48c68-094d-40ee-a818-1c120b134af4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c75ef8131ab47ec860b777df25f6e67'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:52.954861 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-e3c48c68-094d-40ee-a818-1c120b134af4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c75ef8131ab47ec860b777df25f6e67'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:52.955812 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-e3c48c68-094d-40ee-a818-1c120b134af4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c75ef8131ab47ec860b777df25f6e67'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:52.956456 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-e3c48c68-094d-40ee-a818-1c120b134af4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c75ef8131ab47ec860b777df25f6e67'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:52.970278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 956/1926] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:09:52 2022] DELETE /identity/v3/users/0c75ef8131ab47ec860b777df25f6e67 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:52.974769 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4692bc-bd16-449a-a098-68fc7a5c5020 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:52.975239 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4692bc-bd16-449a-a098-68fc7a5c5020 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:52.975322 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4692bc-bd16-449a-a098-68fc7a5c5020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:52.995111 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-ea4692bc-bd16-449a-a098-68fc7a5c5020 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:52.996308 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ea4692bc-bd16-449a-a098-68fc7a5c5020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:53.057268 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 971/1927] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:53.066781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c48c68-094d-40ee-a818-1c120b134af4 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:53.068286 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e3c48c68-094d-40ee-a818-1c120b134af4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:53.133521 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c48c68-094d-40ee-a818-1c120b134af4 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:53.137363 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf12a1fd-935a-44a8-847f-a181ae9fbd24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:53.137974 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12a1fd-935a-44a8-847f-a181ae9fbd24 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:53.138261 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12a1fd-935a-44a8-847f-a181ae9fbd24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:53.138541 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12a1fd-935a-44a8-847f-a181ae9fbd24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:53.148154 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf12a1fd-935a-44a8-847f-a181ae9fbd24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:53.149671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf12a1fd-935a-44a8-847f-a181ae9fbd24 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:53.159213 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 957/1928] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:09:53.278774 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4692bc-bd16-449a-a098-68fc7a5c5020 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:53.280402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ea4692bc-bd16-449a-a098-68fc7a5c5020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:53.292903 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4692bc-bd16-449a-a098-68fc7a5c5020 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:53.299301 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea422d8-e956-4abe-a6c5-4d8b2d9036e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:53.302357 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea422d8-e956-4abe-a6c5-4d8b2d9036e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:53.302357 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea422d8-e956-4abe-a6c5-4d8b2d9036e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:53.302357 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea422d8-e956-4abe-a6c5-4d8b2d9036e3 admin admin] PATH_INFO: `/v3/users/d28f7a2f7130426da0d5a8d1ff943582` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:53.313313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea422d8-e956-4abe-a6c5-4d8b2d9036e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d28f7a2f7130426da0d5a8d1ff943582)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:53.315586 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea422d8-e956-4abe-a6c5-4d8b2d9036e3 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:53.371213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fea422d8-e956-4abe-a6c5-4d8b2d9036e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd28f7a2f7130426da0d5a8d1ff943582'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:53.371964 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fea422d8-e956-4abe-a6c5-4d8b2d9036e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd28f7a2f7130426da0d5a8d1ff943582'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:53.380609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fea422d8-e956-4abe-a6c5-4d8b2d9036e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd28f7a2f7130426da0d5a8d1ff943582'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:53.399124 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fea422d8-e956-4abe-a6c5-4d8b2d9036e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd28f7a2f7130426da0d5a8d1ff943582'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:53.408453 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fea422d8-e956-4abe-a6c5-4d8b2d9036e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd28f7a2f7130426da0d5a8d1ff943582'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:53.416002 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 972/1929] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:09:53 2022] DELETE /identity/v3/users/d28f7a2f7130426da0d5a8d1ff943582 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:09:53.420172 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef255f-0121-4e8e-ae0d-cfbd45c7e4b9 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:53.420565 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef255f-0121-4e8e-ae0d-cfbd45c7e4b9 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:53.420918 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef255f-0121-4e8e-ae0d-cfbd45c7e4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:53.445664 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-82ef255f-0121-4e8e-ae0d-cfbd45c7e4b9 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:53.447685 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-82ef255f-0121-4e8e-ae0d-cfbd45c7e4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:53.469229 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:53.470759 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea422d8-e956-4abe-a6c5-4d8b2d9036e3 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:53.472196 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-fea422d8-e956-4abe-a6c5-4d8b2d9036e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:53.548222 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 958/1930] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:53.556041 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef255f-0121-4e8e-ae0d-cfbd45c7e4b9 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:53.557498 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-82ef255f-0121-4e8e-ae0d-cfbd45c7e4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:53.566508 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea422d8-e956-4abe-a6c5-4d8b2d9036e3 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:53.572472 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6585c1fb-7302-4d4e-9cd1-546407b7f89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:53.573129 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585c1fb-7302-4d4e-9cd1-546407b7f89c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:53.573420 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585c1fb-7302-4d4e-9cd1-546407b7f89c admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:53.573703 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6585c1fb-7302-4d4e-9cd1-546407b7f89c admin admin] PATH_INFO: `/v3/projects/9398746f87a54e39999c7e4d35e60bad` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:53.574840 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6585c1fb-7302-4d4e-9cd1-546407b7f89c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9398746f87a54e39999c7e4d35e60bad)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:53.576790 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6585c1fb-7302-4d4e-9cd1-546407b7f89c admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:53.594324 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:53.642868 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef255f-0121-4e8e-ae0d-cfbd45c7e4b9 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:53.650132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-6585c1fb-7302-4d4e-9cd1-546407b7f89c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9398746f87a54e39999c7e4d35e60bad', 'request_id': 'req-6585c1fb-7302-4d4e-9cd1-546407b7f89c'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:53.660052 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-6585c1fb-7302-4d4e-9cd1-546407b7f89c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9398746f87a54e39999c7e4d35e60bad', 'request_id': 'req-6585c1fb-7302-4d4e-9cd1-546407b7f89c'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:53.666012 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-6585c1fb-7302-4d4e-9cd1-546407b7f89c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9398746f87a54e39999c7e4d35e60bad', 'request_id': 'req-6585c1fb-7302-4d4e-9cd1-546407b7f89c'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:53.679008 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-6585c1fb-7302-4d4e-9cd1-546407b7f89c admin admin] The token cache is being invalidate because project 9398746f87a54e39999c7e4d35e60bad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:09:53.679298 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-6585c1fb-7302-4d4e-9cd1-546407b7f89c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:53.680108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-6585c1fb-7302-4d4e-9cd1-546407b7f89c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:53.682254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 973/1931] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:09:53 2022] DELETE /identity/v3/projects/9398746f87a54e39999c7e4d35e60bad => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:53.687395 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9dc59-0f29-49a1-811b-13821d1c2291 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:53.687780 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9dc59-0f29-49a1-811b-13821d1c2291 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:53.688230 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9dc59-0f29-49a1-811b-13821d1c2291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:53.707425 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fae47-8837-48e9-b02f-4b908364ede9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:53.708215 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fae47-8837-48e9-b02f-4b908364ede9 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:53.708652 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fae47-8837-48e9-b02f-4b908364ede9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:53.708971 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fae47-8837-48e9-b02f-4b908364ede9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:53.710920 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-bc4fae47-8837-48e9-b02f-4b908364ede9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:53.716770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-1be9dc59-0f29-49a1-811b-13821d1c2291 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:53.719789 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-1be9dc59-0f29-49a1-811b-13821d1c2291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:53.812716 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4fae47-8837-48e9-b02f-4b908364ede9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:53.814166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4fae47-8837-48e9-b02f-4b908364ede9 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:53.818694 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 974/1932] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:09:53.828247 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e7e07-9912-4722-903c-e29a6eea74b9 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:53.828696 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e7e07-9912-4722-903c-e29a6eea74b9 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:53.829116 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e7e07-9912-4722-903c-e29a6eea74b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:53.830398 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 959/1933] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:09:53.853424 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-2a4e7e07-9912-4722-903c-e29a6eea74b9 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:53.854676 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2a4e7e07-9912-4722-903c-e29a6eea74b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:53.864245 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 975/1934] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:09:53.868264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fae47-8837-48e9-b02f-4b908364ede9 service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:53.878486 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fae47-8837-48e9-b02f-4b908364ede9 service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:53.883528 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ac793-00dd-4090-8f73-ac9d16fe6abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:53.884128 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ac793-00dd-4090-8f73-ac9d16fe6abd admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:53.884447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ac793-00dd-4090-8f73-ac9d16fe6abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:53.884812 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ac793-00dd-4090-8f73-ac9d16fe6abd admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:53.885680 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ac793-00dd-4090-8f73-ac9d16fe6abd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:09:53.885807 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ac793-00dd-4090-8f73-ac9d16fe6abd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:53.887532 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810ac793-00dd-4090-8f73-ac9d16fe6abd admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:53.900634 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 960/1935] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:09:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:09:53.905049 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e7e07-9912-4722-903c-e29a6eea74b9 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:53.915395 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e7e07-9912-4722-903c-e29a6eea74b9 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:53.922048 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0237c-14af-4b84-9c7f-ef3569968642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:53.923001 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0237c-14af-4b84-9c7f-ef3569968642 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:53.923712 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0237c-14af-4b84-9c7f-ef3569968642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:53.924012 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0237c-14af-4b84-9c7f-ef3569968642 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:53.924747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0237c-14af-4b84-9c7f-ef3569968642 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:53.926795 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0237c-14af-4b84-9c7f-ef3569968642 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:53.957389 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 976/1936] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:09:53 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:09:53.962070 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ac793-00dd-4090-8f73-ac9d16fe6abd admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:53.979491 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ac793-00dd-4090-8f73-ac9d16fe6abd admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:54.003785 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:54.067946 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af122d37-2860-46e2-80cd-53be4941339b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:54.068731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af122d37-2860-46e2-80cd-53be4941339b admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:54.069075 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af122d37-2860-46e2-80cd-53be4941339b admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:54.069353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af122d37-2860-46e2-80cd-53be4941339b admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:54.069974 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af122d37-2860-46e2-80cd-53be4941339b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:54.071914 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af122d37-2860-46e2-80cd-53be4941339b admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:54.105075 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 961/1937] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:09:53 2022] POST /identity/v3/users => generated 292 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:09:54.107084 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0237c-14af-4b84-9c7f-ef3569968642 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:54.120237 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0237c-14af-4b84-9c7f-ef3569968642 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:54.126131 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854cec0-1a0b-4018-a2e6-c0025f922eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:54.126745 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854cec0-1a0b-4018-a2e6-c0025f922eaa admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:54.127022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854cec0-1a0b-4018-a2e6-c0025f922eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:54.127298 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854cec0-1a0b-4018-a2e6-c0025f922eaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:54.127943 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854cec0-1a0b-4018-a2e6-c0025f922eaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:54.129740 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854cec0-1a0b-4018-a2e6-c0025f922eaa admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:54.141153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 977/1938] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:54.146333 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af122d37-2860-46e2-80cd-53be4941339b admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:54.167026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af122d37-2860-46e2-80cd-53be4941339b admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:54.181856 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2543673-032b-4dd3-8315-3afe04351f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:54.183028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2543673-032b-4dd3-8315-3afe04351f54 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:54.183680 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2543673-032b-4dd3-8315-3afe04351f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:54.184254 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2543673-032b-4dd3-8315-3afe04351f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:54.185636 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2543673-032b-4dd3-8315-3afe04351f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:54.189930 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2543673-032b-4dd3-8315-3afe04351f54 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:54.192026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854cec0-1a0b-4018-a2e6-c0025f922eaa admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:54.194918 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-3854cec0-1a0b-4018-a2e6-c0025f922eaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:54.225207 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854cec0-1a0b-4018-a2e6-c0025f922eaa admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:54.237414 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea30a02-b9fd-4d99-80d3-c4a3ca5e2056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:54.239094 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea30a02-b9fd-4d99-80d3-c4a3ca5e2056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:54.240246 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea30a02-b9fd-4d99-80d3-c4a3ca5e2056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:54.240744 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea30a02-b9fd-4d99-80d3-c4a3ca5e2056 admin admin] PATH_INFO: `/v3/projects/61c69232b9d04e1688d00c44f3da647e` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:54.241987 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 962/1939] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:54.243280 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea30a02-b9fd-4d99-80d3-c4a3ca5e2056 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61c69232b9d04e1688d00c44f3da647e)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:54.247709 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2543673-032b-4dd3-8315-3afe04351f54 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:54.248302 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea30a02-b9fd-4d99-80d3-c4a3ca5e2056 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:54.266484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2543673-032b-4dd3-8315-3afe04351f54 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:54.271701 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d7e75-4706-4b17-b70b-dcf4f76b7f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:54.272195 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d7e75-4706-4b17-b70b-dcf4f76b7f3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:54.272475 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d7e75-4706-4b17-b70b-dcf4f76b7f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:54.272779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d7e75-4706-4b17-b70b-dcf4f76b7f3c admin admin] PATH_INFO: `/v3/projects/654274e1a7584c6aabf1e1fa44223f21/users/e1dc2375d02249eb93623f3484585273/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:54.290173 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d7e75-4706-4b17-b70b-dcf4f76b7f3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=654274e1a7584c6aabf1e1fa44223f21, user_id=e1dc2375d02249eb93623f3484585273, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:54.292995 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d7e75-4706-4b17-b70b-dcf4f76b7f3c admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:54.309965 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 963/1940] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:54 2022] PUT /identity/v3/projects/654274e1a7584c6aabf1e1fa44223f21/users/e1dc2375d02249eb93623f3484585273/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:54.320172 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d36a9-63c5-44bb-b75b-b8f5cb4bf9ad None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:54.320469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d36a9-63c5-44bb-b75b-b8f5cb4bf9ad None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:54.320785 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d36a9-63c5-44bb-b75b-b8f5cb4bf9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:54.327043 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-bea30a02-b9fd-4d99-80d3-c4a3ca5e2056 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61c69232b9d04e1688d00c44f3da647e', 'request_id': 'req-bea30a02-b9fd-4d99-80d3-c4a3ca5e2056'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:54.335151 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-bea30a02-b9fd-4d99-80d3-c4a3ca5e2056 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61c69232b9d04e1688d00c44f3da647e', 'request_id': 'req-bea30a02-b9fd-4d99-80d3-c4a3ca5e2056'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:54.337217 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:54.342148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-bea30a02-b9fd-4d99-80d3-c4a3ca5e2056 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61c69232b9d04e1688d00c44f3da647e', 'request_id': 'req-bea30a02-b9fd-4d99-80d3-c4a3ca5e2056'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:54.352562 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-9d1d36a9-63c5-44bb-b75b-b8f5cb4bf9ad None None] MFA Rules not processed for user `e1dc2375d02249eb93623f3484585273`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:54.353777 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9d1d36a9-63c5-44bb-b75b-b8f5cb4bf9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:54.356749 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-bea30a02-b9fd-4d99-80d3-c4a3ca5e2056 admin admin] The token cache is being invalidate because project 61c69232b9d04e1688d00c44f3da647e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:09:54.357633 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-bea30a02-b9fd-4d99-80d3-c4a3ca5e2056 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:54.358290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-bea30a02-b9fd-4d99-80d3-c4a3ca5e2056 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:54.360863 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 978/1941] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:09:54 2022] DELETE /identity/v3/projects/61c69232b9d04e1688d00c44f3da647e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:54.365063 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea30a02-b9fd-4d99-80d3-c4a3ca5e2056 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:54.366467 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-bea30a02-b9fd-4d99-80d3-c4a3ca5e2056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:54.428460 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 964/1942] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:09:54.432848 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d36a9-63c5-44bb-b75b-b8f5cb4bf9ad None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:54.434169 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9d1d36a9-63c5-44bb-b75b-b8f5cb4bf9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:54.434285 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea30a02-b9fd-4d99-80d3-c4a3ca5e2056 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:54.438952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c638013c-4a47-48f7-aa0d-962788e31c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:54.439578 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638013c-4a47-48f7-aa0d-962788e31c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:54.439859 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638013c-4a47-48f7-aa0d-962788e31c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:54.440132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638013c-4a47-48f7-aa0d-962788e31c99 admin admin] PATH_INFO: `/v3/users/3ed0722ac7dc4240b97f805fd67f1731` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:54.441289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c638013c-4a47-48f7-aa0d-962788e31c99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ed0722ac7dc4240b97f805fd67f1731)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:54.443182 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c638013c-4a47-48f7-aa0d-962788e31c99 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:54.447908 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d36a9-63c5-44bb-b75b-b8f5cb4bf9ad None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:54.452859 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8543ca-c483-46cc-9272-edd30e505dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:54.453443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8543ca-c483-46cc-9272-edd30e505dfe admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:54.453727 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8543ca-c483-46cc-9272-edd30e505dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:54.454002 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8543ca-c483-46cc-9272-edd30e505dfe admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:54.454634 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8543ca-c483-46cc-9272-edd30e505dfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:54.456608 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8543ca-c483-46cc-9272-edd30e505dfe admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:54.485644 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 965/1943] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:09:54 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:09:54.489815 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8543ca-c483-46cc-9272-edd30e505dfe admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:54.493273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-c638013c-4a47-48f7-aa0d-962788e31c99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ed0722ac7dc4240b97f805fd67f1731'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:54.493987 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-c638013c-4a47-48f7-aa0d-962788e31c99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ed0722ac7dc4240b97f805fd67f1731'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:54.502102 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-c638013c-4a47-48f7-aa0d-962788e31c99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ed0722ac7dc4240b97f805fd67f1731'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:54.516093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-c638013c-4a47-48f7-aa0d-962788e31c99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ed0722ac7dc4240b97f805fd67f1731'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:54.525538 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-c638013c-4a47-48f7-aa0d-962788e31c99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ed0722ac7dc4240b97f805fd67f1731'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:54.532086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 979/1944] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:09:54 2022] DELETE /identity/v3/users/3ed0722ac7dc4240b97f805fd67f1731 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:09:54.543172 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:54.567770 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8543ca-c483-46cc-9272-edd30e505dfe admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:54.630588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6231ced-e130-45b4-a22f-ce04ed69e79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:54.631311 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6231ced-e130-45b4-a22f-ce04ed69e79e admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:54.631623 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6231ced-e130-45b4-a22f-ce04ed69e79e admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:54.632096 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6231ced-e130-45b4-a22f-ce04ed69e79e admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:54.632774 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6231ced-e130-45b4-a22f-ce04ed69e79e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:54.634682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6231ced-e130-45b4-a22f-ce04ed69e79e admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:54.663382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 966/1945] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:09:54 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:09:54.664747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c638013c-4a47-48f7-aa0d-962788e31c99 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:54.666276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c638013c-4a47-48f7-aa0d-962788e31c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:54.680739 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c638013c-4a47-48f7-aa0d-962788e31c99 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:54.687355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854db13d-1fe6-4a50-8ae4-26993526e125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:54.688049 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854db13d-1fe6-4a50-8ae4-26993526e125 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:54.688329 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854db13d-1fe6-4a50-8ae4-26993526e125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:54.689071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854db13d-1fe6-4a50-8ae4-26993526e125 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:54.689749 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854db13d-1fe6-4a50-8ae4-26993526e125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:54.691572 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854db13d-1fe6-4a50-8ae4-26993526e125 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:54.703966 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 980/1946] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:54.708169 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6231ced-e130-45b4-a22f-ce04ed69e79e admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:54.718681 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6231ced-e130-45b4-a22f-ce04ed69e79e admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:54.724021 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46488a7-ba55-46dd-854d-18ce811a5a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:54.724665 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46488a7-ba55-46dd-854d-18ce811a5a91 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:54.724953 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46488a7-ba55-46dd-854d-18ce811a5a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:54.725235 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46488a7-ba55-46dd-854d-18ce811a5a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:54.725881 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46488a7-ba55-46dd-854d-18ce811a5a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:54.727470 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46488a7-ba55-46dd-854d-18ce811a5a91 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:54.741738 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 967/1947] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:54.743281 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854db13d-1fe6-4a50-8ae4-26993526e125 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:54.756295 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854db13d-1fe6-4a50-8ae4-26993526e125 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:54.763341 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44453ba-dc17-4537-bd38-dc62e86ab973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:54.764098 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44453ba-dc17-4537-bd38-dc62e86ab973 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:54.764347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44453ba-dc17-4537-bd38-dc62e86ab973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:54.764671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44453ba-dc17-4537-bd38-dc62e86ab973 admin admin] PATH_INFO: `/v3/projects/34c2ccc2b2704a47be0f563845e2721b/users/b59a000682b44dce878eb7f775d5e558/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:54.800290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44453ba-dc17-4537-bd38-dc62e86ab973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34c2ccc2b2704a47be0f563845e2721b, user_id=b59a000682b44dce878eb7f775d5e558, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:54.803290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44453ba-dc17-4537-bd38-dc62e86ab973 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:54.839342 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 981/1948] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:54 2022] PUT /identity/v3/projects/34c2ccc2b2704a47be0f563845e2721b/users/b59a000682b44dce878eb7f775d5e558/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:09:54.852860 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46488a7-ba55-46dd-854d-18ce811a5a91 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:54.854663 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b34cb-7145-4525-99a1-50bab7450464 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:54.854781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d46488a7-ba55-46dd-854d-18ce811a5a91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:54.855287 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b34cb-7145-4525-99a1-50bab7450464 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:54.855859 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b34cb-7145-4525-99a1-50bab7450464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:54.878486 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46488a7-ba55-46dd-854d-18ce811a5a91 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:54.922746 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-ac3b34cb-7145-4525-99a1-50bab7450464 None None] MFA Rules not processed for user `b59a000682b44dce878eb7f775d5e558`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:54.923548 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ac3b34cb-7145-4525-99a1-50bab7450464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:54.972554 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:55.018307 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50d052-9a68-4cf7-85c8-43cbd871205b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:55.019334 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50d052-9a68-4cf7-85c8-43cbd871205b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:55.019625 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50d052-9a68-4cf7-85c8-43cbd871205b admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:55.019904 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50d052-9a68-4cf7-85c8-43cbd871205b admin admin] PATH_INFO: `/v3/projects/36d6eab1c640472391f0794ffae48a0d` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:55.024966 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a50d052-9a68-4cf7-85c8-43cbd871205b admin admin] RBAC: Authorizing `identity:delete_project(project_id=36d6eab1c640472391f0794ffae48a0d)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:55.027241 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a50d052-9a68-4cf7-85c8-43cbd871205b admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:55.047010 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 982/1949] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:55.051473 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b34cb-7145-4525-99a1-50bab7450464 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:55.066078 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b34cb-7145-4525-99a1-50bab7450464 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:55.071532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892afff-e3e5-4df4-8c22-d434afa85466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:55.073441 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892afff-e3e5-4df4-8c22-d434afa85466 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:55.073441 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892afff-e3e5-4df4-8c22-d434afa85466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:55.073441 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892afff-e3e5-4df4-8c22-d434afa85466 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:55.073794 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892afff-e3e5-4df4-8c22-d434afa85466 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:55.076418 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892afff-e3e5-4df4-8c22-d434afa85466 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:55.126249 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 983/1950] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:09:55 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:55.129874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1a50d052-9a68-4cf7-85c8-43cbd871205b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36d6eab1c640472391f0794ffae48a0d', 'request_id': 'req-1a50d052-9a68-4cf7-85c8-43cbd871205b'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:55.132715 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892afff-e3e5-4df4-8c22-d434afa85466 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:55.137473 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1a50d052-9a68-4cf7-85c8-43cbd871205b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36d6eab1c640472391f0794ffae48a0d', 'request_id': 'req-1a50d052-9a68-4cf7-85c8-43cbd871205b'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:55.158077 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1a50d052-9a68-4cf7-85c8-43cbd871205b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36d6eab1c640472391f0794ffae48a0d', 'request_id': 'req-1a50d052-9a68-4cf7-85c8-43cbd871205b'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:55.163960 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:55.178140 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1a50d052-9a68-4cf7-85c8-43cbd871205b admin admin] The token cache is being invalidate because project 36d6eab1c640472391f0794ffae48a0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:09:55.179529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1a50d052-9a68-4cf7-85c8-43cbd871205b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:55.180601 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1a50d052-9a68-4cf7-85c8-43cbd871205b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:55.182722 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 968/1951] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:09:54 2022] DELETE /identity/v3/projects/36d6eab1c640472391f0794ffae48a0d => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:55.187757 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55481ca-fb3e-4e9f-9cd8-e977212c55ae None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:55.189069 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55481ca-fb3e-4e9f-9cd8-e977212c55ae None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:55.189370 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55481ca-fb3e-4e9f-9cd8-e977212c55ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:55.229494 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-a55481ca-fb3e-4e9f-9cd8-e977212c55ae None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:55.230648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a55481ca-fb3e-4e9f-9cd8-e977212c55ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:55.253599 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892afff-e3e5-4df4-8c22-d434afa85466 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:55.259250 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57581eec-534f-43c7-b1be-6aba8260fd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:55.259641 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57581eec-534f-43c7-b1be-6aba8260fd9a admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:55.260126 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57581eec-534f-43c7-b1be-6aba8260fd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:55.260473 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57581eec-534f-43c7-b1be-6aba8260fd9a admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:55.261269 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57581eec-534f-43c7-b1be-6aba8260fd9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:55.263638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57581eec-534f-43c7-b1be-6aba8260fd9a admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:55.294165 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 984/1952] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:09:55 2022] POST /identity/v3/users => generated 299 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:55.295782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57581eec-534f-43c7-b1be-6aba8260fd9a admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:55.298699 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-57581eec-534f-43c7-b1be-6aba8260fd9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:55.325518 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57581eec-534f-43c7-b1be-6aba8260fd9a admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:55.325711 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 969/1953] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:55.328944 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d7edc-a63b-4e51-8195-9ac8b0f78740 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:55.329358 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d7edc-a63b-4e51-8195-9ac8b0f78740 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:55.329777 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d7edc-a63b-4e51-8195-9ac8b0f78740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:55.331605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b63836-127c-4813-89f7-d8e2d06459be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:55.332238 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b63836-127c-4813-89f7-d8e2d06459be admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:55.332628 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b63836-127c-4813-89f7-d8e2d06459be admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:55.332976 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b63836-127c-4813-89f7-d8e2d06459be admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:55.333708 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b63836-127c-4813-89f7-d8e2d06459be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:55.335372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b63836-127c-4813-89f7-d8e2d06459be admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:55.351218 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 985/1954] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:09:55.352560 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b63836-127c-4813-89f7-d8e2d06459be admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:55.363005 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-527d7edc-a63b-4e51-8195-9ac8b0f78740 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:55.364726 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-527d7edc-a63b-4e51-8195-9ac8b0f78740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:55.370730 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b63836-127c-4813-89f7-d8e2d06459be admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:55.378931 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 970/1955] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:09:55.382224 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f1582-9312-4396-a22a-8d06e6c77040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:55.383417 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d7edc-a63b-4e51-8195-9ac8b0f78740 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:55.384579 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f1582-9312-4396-a22a-8d06e6c77040 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:55.384871 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f1582-9312-4396-a22a-8d06e6c77040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:55.385545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f1582-9312-4396-a22a-8d06e6c77040 admin admin] PATH_INFO: `/v3/projects/552c0264796e42e9a9a5af5dc288d1b1/users/ff6311f00dab4782b616598dbc99503f/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:55.397110 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d7edc-a63b-4e51-8195-9ac8b0f78740 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:55.403871 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79076b2e-a57f-40ae-a52b-7597a191ab34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:55.404897 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79076b2e-a57f-40ae-a52b-7597a191ab34 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:55.405411 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79076b2e-a57f-40ae-a52b-7597a191ab34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:55.405933 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79076b2e-a57f-40ae-a52b-7597a191ab34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:55.406743 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79076b2e-a57f-40ae-a52b-7597a191ab34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:09:55.407052 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79076b2e-a57f-40ae-a52b-7597a191ab34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:55.408854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79076b2e-a57f-40ae-a52b-7597a191ab34 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:55.412271 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f1582-9312-4396-a22a-8d06e6c77040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=552c0264796e42e9a9a5af5dc288d1b1, user_id=ff6311f00dab4782b616598dbc99503f, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:55.415548 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f1582-9312-4396-a22a-8d06e6c77040 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:55.438587 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 971/1956] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:09:55.442142 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79076b2e-a57f-40ae-a52b-7597a191ab34 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:55.442572 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 986/1957] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:55 2022] PUT /identity/v3/projects/552c0264796e42e9a9a5af5dc288d1b1/users/ff6311f00dab4782b616598dbc99503f/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:09:55.446483 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f1582-9312-4396-a22a-8d06e6c77040 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:55.453734 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79076b2e-a57f-40ae-a52b-7597a191ab34 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:55.530780 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:55.548769 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8deac78-8742-4de6-ae14-50050a126d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:55.549609 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8deac78-8742-4de6-ae14-50050a126d86 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:55.550110 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8deac78-8742-4de6-ae14-50050a126d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:55.550501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8deac78-8742-4de6-ae14-50050a126d86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:55.551393 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8deac78-8742-4de6-ae14-50050a126d86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:55.552801 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f1582-9312-4396-a22a-8d06e6c77040 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:55.554071 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8deac78-8742-4de6-ae14-50050a126d86 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:55.559488 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51a5ae-0382-4859-86dc-c764e02c6cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:55.560455 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51a5ae-0382-4859-86dc-c764e02c6cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:55.560861 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51a5ae-0382-4859-86dc-c764e02c6cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:55.562463 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51a5ae-0382-4859-86dc-c764e02c6cc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:55.563368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51a5ae-0382-4859-86dc-c764e02c6cc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:55.565447 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51a5ae-0382-4859-86dc-c764e02c6cc8 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:55.587050 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 987/1958] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:55.588601 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51a5ae-0382-4859-86dc-c764e02c6cc8 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:55.598068 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 972/1959] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:09:55 2022] POST /identity/v3/projects => generated 373 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:09:55.601005 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51a5ae-0382-4859-86dc-c764e02c6cc8 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:55.602381 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8deac78-8742-4de6-ae14-50050a126d86 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:55.680542 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3bf1d6-bc6e-48b3-9117-682e966f6025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:55.681253 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bf1d6-bc6e-48b3-9117-682e966f6025 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:55.681537 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bf1d6-bc6e-48b3-9117-682e966f6025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:55.681820 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bf1d6-bc6e-48b3-9117-682e966f6025 admin admin] PATH_INFO: `/v3/projects/552c0264796e42e9a9a5af5dc288d1b1/users/ff6311f00dab4782b616598dbc99503f/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:55.683062 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8deac78-8742-4de6-ae14-50050a126d86 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:55.683743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3bf1d6-bc6e-48b3-9117-682e966f6025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=552c0264796e42e9a9a5af5dc288d1b1, user_id=ff6311f00dab4782b616598dbc99503f, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:55.686410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3bf1d6-bc6e-48b3-9117-682e966f6025 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:55.688728 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b43fa-c781-4b3a-af75-35185ba587e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:55.689300 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b43fa-c781-4b3a-af75-35185ba587e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:55.689586 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b43fa-c781-4b3a-af75-35185ba587e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:55.691421 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b43fa-c781-4b3a-af75-35185ba587e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:55.692070 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b43fa-c781-4b3a-af75-35185ba587e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:55.694309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b43fa-c781-4b3a-af75-35185ba587e5 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:55.698738 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 988/1960] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:55 2022] PUT /identity/v3/projects/552c0264796e42e9a9a5af5dc288d1b1/users/ff6311f00dab4782b616598dbc99503f/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:09:55.705220 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:55.711762 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f708823-6393-4dd8-82c2-47456c7ac6d3 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:55.712236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f708823-6393-4dd8-82c2-47456c7ac6d3 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:55.712383 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f708823-6393-4dd8-82c2-47456c7ac6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:55.730563 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 973/1961] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:09:55 2022] POST /identity/v3/users => generated 287 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:55.731426 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b43fa-c781-4b3a-af75-35185ba587e5 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:55.742859 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b43fa-c781-4b3a-af75-35185ba587e5 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:55.760244 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-3f708823-6393-4dd8-82c2-47456c7ac6d3 None None] MFA Rules not processed for user `ff6311f00dab4782b616598dbc99503f`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:55.761718 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-3f708823-6393-4dd8-82c2-47456c7ac6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:55.830833 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e058932-c3ee-4987-9532-a77dc23f5689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:55.832082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e058932-c3ee-4987-9532-a77dc23f5689 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:55.832367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e058932-c3ee-4987-9532-a77dc23f5689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:55.832841 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e058932-c3ee-4987-9532-a77dc23f5689 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:55.833495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e058932-c3ee-4987-9532-a77dc23f5689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:55.835075 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e058932-c3ee-4987-9532-a77dc23f5689 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:55.840799 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 989/1962] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:09:55.848002 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 974/1963] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:09:55.853447 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f708823-6393-4dd8-82c2-47456c7ac6d3 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:55.855573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e058932-c3ee-4987-9532-a77dc23f5689 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:55.857895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-3e058932-c3ee-4987-9532-a77dc23f5689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:55.866032 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f708823-6393-4dd8-82c2-47456c7ac6d3 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:55.883602 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310568c4-95ae-4043-aba8-af633c981cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:55.884242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310568c4-95ae-4043-aba8-af633c981cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:55.884683 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310568c4-95ae-4043-aba8-af633c981cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:55.884962 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310568c4-95ae-4043-aba8-af633c981cc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:55.885754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310568c4-95ae-4043-aba8-af633c981cc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:55.887366 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310568c4-95ae-4043-aba8-af633c981cc8 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:55.901182 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 990/1964] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:55.902194 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310568c4-95ae-4043-aba8-af633c981cc8 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:55.917498 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310568c4-95ae-4043-aba8-af633c981cc8 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:55.922731 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15abbf-2cfd-480a-b820-b214c7cc2f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:55.923412 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15abbf-2cfd-480a-b820-b214c7cc2f3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:55.923697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15abbf-2cfd-480a-b820-b214c7cc2f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:55.923973 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15abbf-2cfd-480a-b820-b214c7cc2f3f admin admin] PATH_INFO: `/v3/projects/bcef7335dafe4b29960c12ce5ab5330b/users/63286ecd4376471f8ab5f647674eaf43/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:55.935639 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e058932-c3ee-4987-9532-a77dc23f5689 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:55.939989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15abbf-2cfd-480a-b820-b214c7cc2f3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcef7335dafe4b29960c12ce5ab5330b, user_id=63286ecd4376471f8ab5f647674eaf43, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:55.940113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984cda0d-327e-4d57-abdb-a7a0ea238d59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:55.940577 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984cda0d-327e-4d57-abdb-a7a0ea238d59 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:55.940862 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984cda0d-327e-4d57-abdb-a7a0ea238d59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:55.941136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984cda0d-327e-4d57-abdb-a7a0ea238d59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:55.942902 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-984cda0d-327e-4d57-abdb-a7a0ea238d59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:55.943080 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15abbf-2cfd-480a-b820-b214c7cc2f3f admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:55.964522 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 991/1965] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:55 2022] PUT /identity/v3/projects/bcef7335dafe4b29960c12ce5ab5330b/users/63286ecd4376471f8ab5f647674eaf43/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:55.966536 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fb446-13bb-4ccd-8f94-31cad7f5cb83 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:55.966752 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fb446-13bb-4ccd-8f94-31cad7f5cb83 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:55.967257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fb446-13bb-4ccd-8f94-31cad7f5cb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:55.990119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-8a6fb446-13bb-4ccd-8f94-31cad7f5cb83 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:55.991343 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8a6fb446-13bb-4ccd-8f94-31cad7f5cb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:56.015079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984cda0d-327e-4d57-abdb-a7a0ea238d59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:56.016554 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984cda0d-327e-4d57-abdb-a7a0ea238d59 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:56.036451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:56.038206 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 975/1966] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:09:56.076718 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 992/1967] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:56.091489 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984cda0d-327e-4d57-abdb-a7a0ea238d59 service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:56.095833 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-984cda0d-327e-4d57-abdb-a7a0ea238d59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:56.175449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:56.209767 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984cda0d-327e-4d57-abdb-a7a0ea238d59 service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:56.324249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8245b18-0fb3-4a88-a57d-81984350db08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:56.324912 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8245b18-0fb3-4a88-a57d-81984350db08 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:56.325227 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8245b18-0fb3-4a88-a57d-81984350db08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:56.325505 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8245b18-0fb3-4a88-a57d-81984350db08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:56.346122 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8245b18-0fb3-4a88-a57d-81984350db08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:56.348967 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8245b18-0fb3-4a88-a57d-81984350db08 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:56.368710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 976/1968] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:09:56.548019 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fb446-13bb-4ccd-8f94-31cad7f5cb83 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:56.554404 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8a6fb446-13bb-4ccd-8f94-31cad7f5cb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:56.570985 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fb446-13bb-4ccd-8f94-31cad7f5cb83 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:56.575956 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4bb9b3-5b35-4081-b5a9-787d347ea41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:56.576644 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4bb9b3-5b35-4081-b5a9-787d347ea41d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:56.576931 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4bb9b3-5b35-4081-b5a9-787d347ea41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:56.577284 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4bb9b3-5b35-4081-b5a9-787d347ea41d admin admin] PATH_INFO: `/v3/users/7e0cd8a9bb5442cc84130943983d0f36` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:56.579345 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4bb9b3-5b35-4081-b5a9-787d347ea41d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e0cd8a9bb5442cc84130943983d0f36)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:56.581472 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4bb9b3-5b35-4081-b5a9-787d347ea41d admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:56.644885 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-dd4bb9b3-5b35-4081-b5a9-787d347ea41d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e0cd8a9bb5442cc84130943983d0f36'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:56.646037 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-dd4bb9b3-5b35-4081-b5a9-787d347ea41d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e0cd8a9bb5442cc84130943983d0f36'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:56.657765 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-dd4bb9b3-5b35-4081-b5a9-787d347ea41d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e0cd8a9bb5442cc84130943983d0f36'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:56.682343 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-dd4bb9b3-5b35-4081-b5a9-787d347ea41d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e0cd8a9bb5442cc84130943983d0f36'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:56.698066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-dd4bb9b3-5b35-4081-b5a9-787d347ea41d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e0cd8a9bb5442cc84130943983d0f36'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:56.707345 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 993/1969] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:09:56 2022] DELETE /identity/v3/users/7e0cd8a9bb5442cc84130943983d0f36 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:56.711914 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e7233-b90d-48b3-bed9-abd813b62ec3 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:56.712533 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e7233-b90d-48b3-bed9-abd813b62ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:56.712836 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e7233-b90d-48b3-bed9-abd813b62ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:56.745839 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-ba8e7233-b90d-48b3-bed9-abd813b62ec3 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:56.749055 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ba8e7233-b90d-48b3-bed9-abd813b62ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:56.923800 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 977/1970] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:56.935907 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4bb9b3-5b35-4081-b5a9-787d347ea41d admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:56.939327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-dd4bb9b3-5b35-4081-b5a9-787d347ea41d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:57.034762 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4bb9b3-5b35-4081-b5a9-787d347ea41d admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:57.038989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83828e4-26e9-44c3-bb98-b67c858b127e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:57.039741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83828e4-26e9-44c3-bb98-b67c858b127e service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:57.040868 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83828e4-26e9-44c3-bb98-b67c858b127e service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:57.040868 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83828e4-26e9-44c3-bb98-b67c858b127e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:57.065108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83828e4-26e9-44c3-bb98-b67c858b127e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:57.066708 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83828e4-26e9-44c3-bb98-b67c858b127e service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:57.089899 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 994/1971] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:09:57.269509 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e7233-b90d-48b3-bed9-abd813b62ec3 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:57.270902 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ba8e7233-b90d-48b3-bed9-abd813b62ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:57.289150 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e7233-b90d-48b3-bed9-abd813b62ec3 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:57.305142 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3a452-1ff0-49b9-87a1-545d8074abd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:57.305816 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3a452-1ff0-49b9-87a1-545d8074abd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:57.306104 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3a452-1ff0-49b9-87a1-545d8074abd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:57.306384 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3a452-1ff0-49b9-87a1-545d8074abd2 admin admin] PATH_INFO: `/v3/users/08480a18e2f9491382d87daa3c3799ad` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:57.308353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3a452-1ff0-49b9-87a1-545d8074abd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08480a18e2f9491382d87daa3c3799ad)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:57.310381 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3a452-1ff0-49b9-87a1-545d8074abd2 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:57.414442 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-17e3a452-1ff0-49b9-87a1-545d8074abd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08480a18e2f9491382d87daa3c3799ad'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:57.425453 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-17e3a452-1ff0-49b9-87a1-545d8074abd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08480a18e2f9491382d87daa3c3799ad'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:57.442820 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-17e3a452-1ff0-49b9-87a1-545d8074abd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08480a18e2f9491382d87daa3c3799ad'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:57.443744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-17e3a452-1ff0-49b9-87a1-545d8074abd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08480a18e2f9491382d87daa3c3799ad'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:57.444475 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-17e3a452-1ff0-49b9-87a1-545d8074abd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08480a18e2f9491382d87daa3c3799ad'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:57.476801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 978/1972] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:09:57 2022] DELETE /identity/v3/users/08480a18e2f9491382d87daa3c3799ad => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:57.477092 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0d06e-7a66-42bb-be3a-f0b82fd31bd2 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:57.477340 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0d06e-7a66-42bb-be3a-f0b82fd31bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:57.477624 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0d06e-7a66-42bb-be3a-f0b82fd31bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:57.503182 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-21d0d06e-7a66-42bb-be3a-f0b82fd31bd2 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:57.503728 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-21d0d06e-7a66-42bb-be3a-f0b82fd31bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:57.614057 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:57.661316 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 995/1973] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:57.675527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3a452-1ff0-49b9-87a1-545d8074abd2 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:57.678585 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-17e3a452-1ff0-49b9-87a1-545d8074abd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:57.736366 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:57.747954 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9eb84f-5e2d-49c9-8a0b-ebd8ba07a61d None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:57.748336 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9eb84f-5e2d-49c9-8a0b-ebd8ba07a61d None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:57.748784 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9eb84f-5e2d-49c9-8a0b-ebd8ba07a61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:57.797348 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-2b9eb84f-5e2d-49c9-8a0b-ebd8ba07a61d None None] MFA Rules not processed for user `63286ecd4376471f8ab5f647674eaf43`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:57.800577 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2b9eb84f-5e2d-49c9-8a0b-ebd8ba07a61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:57.905249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3a452-1ff0-49b9-87a1-545d8074abd2 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:57.923318 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052d71c-f558-4c93-831b-64cb186f34d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:57.924212 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052d71c-f558-4c93-831b-64cb186f34d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:57.924747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052d71c-f558-4c93-831b-64cb186f34d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:57.926302 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052d71c-f558-4c93-831b-64cb186f34d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:57.940805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052d71c-f558-4c93-831b-64cb186f34d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:57.949745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052d71c-f558-4c93-831b-64cb186f34d9 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:57.957781 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 996/1974] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:57.958681 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9eb84f-5e2d-49c9-8a0b-ebd8ba07a61d None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:57.960028 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2b9eb84f-5e2d-49c9-8a0b-ebd8ba07a61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:57.981538 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 979/1975] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:09:57.982543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052d71c-f558-4c93-831b-64cb186f34d9 service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:57.985498 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f052d71c-f558-4c93-831b-64cb186f34d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:58.031280 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9eb84f-5e2d-49c9-8a0b-ebd8ba07a61d None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:58.036782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ff2bf-dfe6-4718-b75a-8ee8d9be37ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:58.037542 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ff2bf-dfe6-4718-b75a-8ee8d9be37ad service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:58.037926 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ff2bf-dfe6-4718-b75a-8ee8d9be37ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:58.038569 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ff2bf-dfe6-4718-b75a-8ee8d9be37ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:58.042470 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8c9ff2bf-dfe6-4718-b75a-8ee8d9be37ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:58.107957 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052d71c-f558-4c93-831b-64cb186f34d9 service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:58.129030 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9fc83b-aa5d-40ce-81ee-a445e42432d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:58.129671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9fc83b-aa5d-40ce-81ee-a445e42432d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:58.129954 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9fc83b-aa5d-40ce-81ee-a445e42432d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:58.130233 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9fc83b-aa5d-40ce-81ee-a445e42432d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:58.150557 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9fc83b-aa5d-40ce-81ee-a445e42432d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:58.153519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9fc83b-aa5d-40ce-81ee-a445e42432d7 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:58.169869 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 980/1976] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:09:58.201569 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ff2bf-dfe6-4718-b75a-8ee8d9be37ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:58.203500 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ff2bf-dfe6-4718-b75a-8ee8d9be37ad service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:58.229553 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 997/1977] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:09:58.451747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9fc83b-aa5d-40ce-81ee-a445e42432d7 service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:58.456080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-0f9fc83b-aa5d-40ce-81ee-a445e42432d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:58.505649 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9fc83b-aa5d-40ce-81ee-a445e42432d7 service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:58.517586 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf44fbf-11c3-450e-bbae-e8f01477b933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:58.518619 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf44fbf-11c3-450e-bbae-e8f01477b933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:58.519821 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf44fbf-11c3-450e-bbae-e8f01477b933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:58.520144 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf44fbf-11c3-450e-bbae-e8f01477b933 admin admin] PATH_INFO: `/v3/projects/22784ba66b5247b880f3bf88ae2a900e` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:58.522640 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf44fbf-11c3-450e-bbae-e8f01477b933 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22784ba66b5247b880f3bf88ae2a900e)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:58.525778 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf44fbf-11c3-450e-bbae-e8f01477b933 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:58.654479 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ff2bf-dfe6-4718-b75a-8ee8d9be37ad service cinder] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:58.657264 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8c9ff2bf-dfe6-4718-b75a-8ee8d9be37ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:58.664389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-0bf44fbf-11c3-450e-bbae-e8f01477b933 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22784ba66b5247b880f3bf88ae2a900e', 'request_id': 'req-0bf44fbf-11c3-450e-bbae-e8f01477b933'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:58.696972 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ff2bf-dfe6-4718-b75a-8ee8d9be37ad service cinder] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:58.698713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-0bf44fbf-11c3-450e-bbae-e8f01477b933 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22784ba66b5247b880f3bf88ae2a900e', 'request_id': 'req-0bf44fbf-11c3-450e-bbae-e8f01477b933'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:58.719411 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-0bf44fbf-11c3-450e-bbae-e8f01477b933 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22784ba66b5247b880f3bf88ae2a900e', 'request_id': 'req-0bf44fbf-11c3-450e-bbae-e8f01477b933'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:58.779621 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-0bf44fbf-11c3-450e-bbae-e8f01477b933 admin admin] The token cache is being invalidate because project 22784ba66b5247b880f3bf88ae2a900e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:09:58.779936 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-0bf44fbf-11c3-450e-bbae-e8f01477b933 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:58.781129 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-0bf44fbf-11c3-450e-bbae-e8f01477b933 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:58.785779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 981/1978] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:09:58 2022] DELETE /identity/v3/projects/22784ba66b5247b880f3bf88ae2a900e => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:58.794880 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765aebae-4b55-4f38-9edd-3520c98a7677 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:58.798013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765aebae-4b55-4f38-9edd-3520c98a7677 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:58.798564 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765aebae-4b55-4f38-9edd-3520c98a7677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:58.821615 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:58.871521 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-765aebae-4b55-4f38-9edd-3520c98a7677 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:58.873356 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-765aebae-4b55-4f38-9edd-3520c98a7677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:58.881640 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80122c2f-62e1-47bf-b8e7-3e8e5b48f6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:58.882298 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80122c2f-62e1-47bf-b8e7-3e8e5b48f6d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:58.883003 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80122c2f-62e1-47bf-b8e7-3e8e5b48f6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:58.883382 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80122c2f-62e1-47bf-b8e7-3e8e5b48f6d5 admin admin] PATH_INFO: `/v3/projects/5175d054cb01475ab24f91b354ab7011` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:58.884948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80122c2f-62e1-47bf-b8e7-3e8e5b48f6d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5175d054cb01475ab24f91b354ab7011)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:58.888261 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80122c2f-62e1-47bf-b8e7-3e8e5b48f6d5 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:58.888411 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 982/1979] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:58.897206 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b087f1-d071-48b5-94bd-f6bdd4576fa2 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:58.897563 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b087f1-d071-48b5-94bd-f6bdd4576fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:58.897908 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b087f1-d071-48b5-94bd-f6bdd4576fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:58.957346 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-f3b087f1-d071-48b5-94bd-f6bdd4576fa2 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:58.958769 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f3b087f1-d071-48b5-94bd-f6bdd4576fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:58.979478 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-80122c2f-62e1-47bf-b8e7-3e8e5b48f6d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5175d054cb01475ab24f91b354ab7011', 'request_id': 'req-80122c2f-62e1-47bf-b8e7-3e8e5b48f6d5'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:58.994019 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-80122c2f-62e1-47bf-b8e7-3e8e5b48f6d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5175d054cb01475ab24f91b354ab7011', 'request_id': 'req-80122c2f-62e1-47bf-b8e7-3e8e5b48f6d5'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:59.001400 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:59.005808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-80122c2f-62e1-47bf-b8e7-3e8e5b48f6d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5175d054cb01475ab24f91b354ab7011', 'request_id': 'req-80122c2f-62e1-47bf-b8e7-3e8e5b48f6d5'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:59.019992 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-80122c2f-62e1-47bf-b8e7-3e8e5b48f6d5 admin admin] The token cache is being invalidate because project 5175d054cb01475ab24f91b354ab7011 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:09:59.020279 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-80122c2f-62e1-47bf-b8e7-3e8e5b48f6d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:59.020969 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-80122c2f-62e1-47bf-b8e7-3e8e5b48f6d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:59.022918 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 998/1980] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:09:58 2022] DELETE /identity/v3/projects/5175d054cb01475ab24f91b354ab7011 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:59.027661 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80122c2f-62e1-47bf-b8e7-3e8e5b48f6d5 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:59.030873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-80122c2f-62e1-47bf-b8e7-3e8e5b48f6d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:59.055655 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 983/1981] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:59.057010 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b087f1-d071-48b5-94bd-f6bdd4576fa2 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:59.070417 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b087f1-d071-48b5-94bd-f6bdd4576fa2 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:59.077519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b8397-2aaa-4019-b1a4-9454d390763c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:59.078244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b8397-2aaa-4019-b1a4-9454d390763c admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:59.078594 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b8397-2aaa-4019-b1a4-9454d390763c admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:59.079032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b8397-2aaa-4019-b1a4-9454d390763c admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:59.079818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b8397-2aaa-4019-b1a4-9454d390763c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:09:59.080236 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b8397-2aaa-4019-b1a4-9454d390763c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:59.082091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b8397-2aaa-4019-b1a4-9454d390763c admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:59.104185 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 984/1982] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:09:59.111024 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b8397-2aaa-4019-b1a4-9454d390763c admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:59.122042 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b8397-2aaa-4019-b1a4-9454d390763c admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:59.131416 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008a16b-db5c-4ea9-acfe-cf1cb605f7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:59.132047 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008a16b-db5c-4ea9-acfe-cf1cb605f7e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:59.132334 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008a16b-db5c-4ea9-acfe-cf1cb605f7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:59.132649 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008a16b-db5c-4ea9-acfe-cf1cb605f7e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:59.133410 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7008a16b-db5c-4ea9-acfe-cf1cb605f7e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:59.135614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7008a16b-db5c-4ea9-acfe-cf1cb605f7e0 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:59.155650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80122c2f-62e1-47bf-b8e7-3e8e5b48f6d5 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:59.164274 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c28ea8-7fcf-4c56-85df-0b1ad38fb90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:59.164274 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c28ea8-7fcf-4c56-85df-0b1ad38fb90a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:59.164274 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c28ea8-7fcf-4c56-85df-0b1ad38fb90a admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:59.164274 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c28ea8-7fcf-4c56-85df-0b1ad38fb90a admin admin] PATH_INFO: `/v3/users/2e15fcb714be486e99d0905b3d88fc6f` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:59.170078 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c28ea8-7fcf-4c56-85df-0b1ad38fb90a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e15fcb714be486e99d0905b3d88fc6f)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:59.173107 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c28ea8-7fcf-4c56-85df-0b1ad38fb90a admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:59.189992 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 985/1983] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:09:59 2022] POST /identity/v3/projects => generated 381 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:59.195770 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008a16b-db5c-4ea9-acfe-cf1cb605f7e0 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:59.272354 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-48c28ea8-7fcf-4c56-85df-0b1ad38fb90a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e15fcb714be486e99d0905b3d88fc6f'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:59.273072 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-48c28ea8-7fcf-4c56-85df-0b1ad38fb90a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e15fcb714be486e99d0905b3d88fc6f'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:59.285589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-48c28ea8-7fcf-4c56-85df-0b1ad38fb90a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e15fcb714be486e99d0905b3d88fc6f'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:59.295689 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:59.307058 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-48c28ea8-7fcf-4c56-85df-0b1ad38fb90a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e15fcb714be486e99d0905b3d88fc6f'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:59.308583 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008a16b-db5c-4ea9-acfe-cf1cb605f7e0 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:59.315721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15e3ac-dfa2-4dd7-8ac5-a9f8a87e387c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:59.316467 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15e3ac-dfa2-4dd7-8ac5-a9f8a87e387c admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:59.316863 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15e3ac-dfa2-4dd7-8ac5-a9f8a87e387c admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:59.317229 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15e3ac-dfa2-4dd7-8ac5-a9f8a87e387c admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:59.317708 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-48c28ea8-7fcf-4c56-85df-0b1ad38fb90a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e15fcb714be486e99d0905b3d88fc6f'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:09:59.318188 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15e3ac-dfa2-4dd7-8ac5-a9f8a87e387c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:59.324952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 999/1984] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:09:59 2022] DELETE /identity/v3/users/2e15fcb714be486e99d0905b3d88fc6f => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:09:59.327478 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15e3ac-dfa2-4dd7-8ac5-a9f8a87e387c admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:59.329870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c28ea8-7fcf-4c56-85df-0b1ad38fb90a admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:59.331396 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-48c28ea8-7fcf-4c56-85df-0b1ad38fb90a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:59.359595 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 986/1985] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:09:59 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:09:59.361432 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15e3ac-dfa2-4dd7-8ac5-a9f8a87e387c admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:59.363438 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-0e15e3ac-dfa2-4dd7-8ac5-a9f8a87e387c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:59.378894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15e3ac-dfa2-4dd7-8ac5-a9f8a87e387c admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:59.404461 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c28ea8-7fcf-4c56-85df-0b1ad38fb90a admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:59.408817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b08fd-f144-413f-86f4-1a29a3b2464a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:59.409310 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b08fd-f144-413f-86f4-1a29a3b2464a service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:59.409598 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b08fd-f144-413f-86f4-1a29a3b2464a service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:59.409960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b08fd-f144-413f-86f4-1a29a3b2464a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:59.411803 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f09b08fd-f144-413f-86f4-1a29a3b2464a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:59.449788 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f84f99-bd84-46ed-9eee-910460873655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:59.450606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f84f99-bd84-46ed-9eee-910460873655 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:59.450764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f84f99-bd84-46ed-9eee-910460873655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:59.451099 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f84f99-bd84-46ed-9eee-910460873655 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:59.451763 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f84f99-bd84-46ed-9eee-910460873655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:59.454026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f84f99-bd84-46ed-9eee-910460873655 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:59.467115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 987/1986] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:09:59.472562 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f84f99-bd84-46ed-9eee-910460873655 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:59.486527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f84f99-bd84-46ed-9eee-910460873655 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:59.495998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa04e8b-4e5c-40cb-a271-c2044fe51b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:59.497259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa04e8b-4e5c-40cb-a271-c2044fe51b1a admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:59.497548 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa04e8b-4e5c-40cb-a271-c2044fe51b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:59.497942 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa04e8b-4e5c-40cb-a271-c2044fe51b1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:59.500166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa04e8b-4e5c-40cb-a271-c2044fe51b1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:59.503309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa04e8b-4e5c-40cb-a271-c2044fe51b1a admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:59.516810 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 988/1987] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:09:59.522303 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa04e8b-4e5c-40cb-a271-c2044fe51b1a admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:59.529276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b08fd-f144-413f-86f4-1a29a3b2464a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:59.531991 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b08fd-f144-413f-86f4-1a29a3b2464a service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:59.542273 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa04e8b-4e5c-40cb-a271-c2044fe51b1a admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:59.546310 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1000/1988] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:09:59.570183 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ee5f7-d7cb-4f2a-bb9d-94bccca05bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:59.570905 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ee5f7-d7cb-4f2a-bb9d-94bccca05bae admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:59.571232 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ee5f7-d7cb-4f2a-bb9d-94bccca05bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:59.571532 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ee5f7-d7cb-4f2a-bb9d-94bccca05bae admin admin] PATH_INFO: `/v3/projects/10a83daa0cad4d0fb210247c411e4c0a/users/fb594a081d194e709023dc7c557afe4c/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:59.597351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:09:59.616450 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ee5f7-d7cb-4f2a-bb9d-94bccca05bae admin admin] RBAC: Authorizing `identity:create_grant(project_id=10a83daa0cad4d0fb210247c411e4c0a, user_id=fb594a081d194e709023dc7c557afe4c, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:59.619807 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ee5f7-d7cb-4f2a-bb9d-94bccca05bae admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:09:59.653174 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 989/1989] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:09:59 2022] PUT /identity/v3/projects/10a83daa0cad4d0fb210247c411e4c0a/users/fb594a081d194e709023dc7c557afe4c/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:09:59.658023 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1163e-9a24-49c0-9ae2-5a76eeb304cc None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:59.658524 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1163e-9a24-49c0-9ae2-5a76eeb304cc None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:59.658633 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1163e-9a24-49c0-9ae2-5a76eeb304cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:59.690589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-c9b1163e-9a24-49c0-9ae2-5a76eeb304cc None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:09:59.692648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c9b1163e-9a24-49c0-9ae2-5a76eeb304cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:59.801561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1001/1990] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:09:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:09:59.814541 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ee5f7-d7cb-4f2a-bb9d-94bccca05bae admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:09:59.818362 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-fc9ee5f7-d7cb-4f2a-bb9d-94bccca05bae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:09:59.954169 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ee5f7-d7cb-4f2a-bb9d-94bccca05bae admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:09:59.967393 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56812ba-5655-42cb-b5b1-d84bf0a761fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:09:59.968199 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56812ba-5655-42cb-b5b1-d84bf0a761fb service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:09:59.968466 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56812ba-5655-42cb-b5b1-d84bf0a761fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:09:59.968695 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56812ba-5655-42cb-b5b1-d84bf0a761fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:09:59.991321 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56812ba-5655-42cb-b5b1-d84bf0a761fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:09:59.992854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56812ba-5655-42cb-b5b1-d84bf0a761fb service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:00.005000 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55e603-c03e-4de4-93ba-36ace16a5dcc None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:00.005192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55e603-c03e-4de4-93ba-36ace16a5dcc None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:00.005448 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55e603-c03e-4de4-93ba-36ace16a5dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:00.016364 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 990/1991] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:09:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:10:00.033757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-fc55e603-c03e-4de4-93ba-36ace16a5dcc None None] MFA Rules not processed for user `63286ecd4376471f8ab5f647674eaf43`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:00.035089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-fc55e603-c03e-4de4-93ba-36ace16a5dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:00.076387 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:00.142227 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1002/1992] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:10:00.187627 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56812ba-5655-42cb-b5b1-d84bf0a761fb service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:00.191926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e56812ba-5655-42cb-b5b1-d84bf0a761fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:00.211287 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56812ba-5655-42cb-b5b1-d84bf0a761fb service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:00.219134 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56aa72d-d6e1-4210-8b68-f040178d9391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:00.220218 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56aa72d-d6e1-4210-8b68-f040178d9391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:00.220693 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56aa72d-d6e1-4210-8b68-f040178d9391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:00.221188 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56aa72d-d6e1-4210-8b68-f040178d9391 admin admin] PATH_INFO: `/v3/projects/fa872cc55f5c4953a1f469934f3ed115` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:00.222836 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56aa72d-d6e1-4210-8b68-f040178d9391 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa872cc55f5c4953a1f469934f3ed115)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:00.225743 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56aa72d-d6e1-4210-8b68-f040178d9391 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:00.306917 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f56aa72d-d6e1-4210-8b68-f040178d9391 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa872cc55f5c4953a1f469934f3ed115', 'request_id': 'req-f56aa72d-d6e1-4210-8b68-f040178d9391'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:00.316816 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f56aa72d-d6e1-4210-8b68-f040178d9391 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa872cc55f5c4953a1f469934f3ed115', 'request_id': 'req-f56aa72d-d6e1-4210-8b68-f040178d9391'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:00.328611 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f56aa72d-d6e1-4210-8b68-f040178d9391 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa872cc55f5c4953a1f469934f3ed115', 'request_id': 'req-f56aa72d-d6e1-4210-8b68-f040178d9391'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:00.342102 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f56aa72d-d6e1-4210-8b68-f040178d9391 admin admin] The token cache is being invalidate because project fa872cc55f5c4953a1f469934f3ed115 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:10:00.342398 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f56aa72d-d6e1-4210-8b68-f040178d9391 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:00.345286 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f56aa72d-d6e1-4210-8b68-f040178d9391 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:00.347858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 991/1993] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:10:00 2022] DELETE /identity/v3/projects/fa872cc55f5c4953a1f469934f3ed115 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:10:00.352752 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55e603-c03e-4de4-93ba-36ace16a5dcc None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:00.366967 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55e603-c03e-4de4-93ba-36ace16a5dcc None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:00.452089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0a900-2b84-44da-a9b3-327bef32c692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:00.452574 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0a900-2b84-44da-a9b3-327bef32c692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:00.453099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0a900-2b84-44da-a9b3-327bef32c692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:00.453307 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0a900-2b84-44da-a9b3-327bef32c692 admin admin] PATH_INFO: `/v3/users/e33c048e2d1842348db0a6c8a7707cd2` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:00.454522 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0a900-2b84-44da-a9b3-327bef32c692 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e33c048e2d1842348db0a6c8a7707cd2)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:00.456822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0a900-2b84-44da-a9b3-327bef32c692 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:00.531638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-eaa0a900-2b84-44da-a9b3-327bef32c692 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e33c048e2d1842348db0a6c8a7707cd2'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:00.532470 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-eaa0a900-2b84-44da-a9b3-327bef32c692 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e33c048e2d1842348db0a6c8a7707cd2'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:00.540783 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-eaa0a900-2b84-44da-a9b3-327bef32c692 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e33c048e2d1842348db0a6c8a7707cd2'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:00.561293 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-eaa0a900-2b84-44da-a9b3-327bef32c692 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e33c048e2d1842348db0a6c8a7707cd2'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:00.574382 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-eaa0a900-2b84-44da-a9b3-327bef32c692 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e33c048e2d1842348db0a6c8a7707cd2'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:00.585164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1003/1994] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:10:00 2022] DELETE /identity/v3/users/e33c048e2d1842348db0a6c8a7707cd2 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:10:00.980373 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56aa72d-d6e1-4210-8b68-f040178d9391 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:00.981910 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f56aa72d-d6e1-4210-8b68-f040178d9391 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:01.006524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56aa72d-d6e1-4210-8b68-f040178d9391 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:01.125970 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79831c-9290-4597-943e-eaa62183ca3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:01.127038 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79831c-9290-4597-943e-eaa62183ca3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:01.127595 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79831c-9290-4597-943e-eaa62183ca3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:01.128130 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79831c-9290-4597-943e-eaa62183ca3b admin admin] PATH_INFO: `/v3/projects/57a110d5b8654694b6f8e21085c7b287` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:01.129744 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79831c-9290-4597-943e-eaa62183ca3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=57a110d5b8654694b6f8e21085c7b287)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:01.131874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79831c-9290-4597-943e-eaa62183ca3b admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:01.151000 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:01.214982 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-3c79831c-9290-4597-943e-eaa62183ca3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57a110d5b8654694b6f8e21085c7b287', 'request_id': 'req-3c79831c-9290-4597-943e-eaa62183ca3b'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:01.226655 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-3c79831c-9290-4597-943e-eaa62183ca3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57a110d5b8654694b6f8e21085c7b287', 'request_id': 'req-3c79831c-9290-4597-943e-eaa62183ca3b'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:01.237934 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-3c79831c-9290-4597-943e-eaa62183ca3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57a110d5b8654694b6f8e21085c7b287', 'request_id': 'req-3c79831c-9290-4597-943e-eaa62183ca3b'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:01.253727 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-3c79831c-9290-4597-943e-eaa62183ca3b admin admin] The token cache is being invalidate because project 57a110d5b8654694b6f8e21085c7b287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:10:01.254138 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-3c79831c-9290-4597-943e-eaa62183ca3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:01.254989 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-3c79831c-9290-4597-943e-eaa62183ca3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:01.256973 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 992/1995] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:10:00 2022] DELETE /identity/v3/projects/57a110d5b8654694b6f8e21085c7b287 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:10:01.262148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e139e9-c06b-4e93-845f-336345c8054e None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:01.267209 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e139e9-c06b-4e93-845f-336345c8054e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:01.268918 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e139e9-c06b-4e93-845f-336345c8054e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:01.326469 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-32e139e9-c06b-4e93-845f-336345c8054e None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:01.327834 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-32e139e9-c06b-4e93-845f-336345c8054e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:01.422548 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:01.437756 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1004/1996] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:01.446294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e411239-4220-4f4d-9fba-8305f9d28cf9 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:01.446595 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e411239-4220-4f4d-9fba-8305f9d28cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:01.446874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e411239-4220-4f4d-9fba-8305f9d28cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:01.485613 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e139e9-c06b-4e93-845f-336345c8054e None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:01.487862 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-32e139e9-c06b-4e93-845f-336345c8054e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:01.493332 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1e411239-4220-4f4d-9fba-8305f9d28cf9 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:01.494781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1e411239-4220-4f4d-9fba-8305f9d28cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:01.516078 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e139e9-c06b-4e93-845f-336345c8054e None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:01.519979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 993/1997] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:01.521512 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e411239-4220-4f4d-9fba-8305f9d28cf9 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:01.526769 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551695a-9de4-45ea-9d20-44a938b4bf23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:01.527654 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551695a-9de4-45ea-9d20-44a938b4bf23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:01.528167 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551695a-9de4-45ea-9d20-44a938b4bf23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:01.528793 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551695a-9de4-45ea-9d20-44a938b4bf23 admin admin] PATH_INFO: `/v3/users/127eff42dfb141c3b78ce04636af1f28` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:01.530510 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2551695a-9de4-45ea-9d20-44a938b4bf23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=127eff42dfb141c3b78ce04636af1f28)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:01.532821 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e411239-4220-4f4d-9fba-8305f9d28cf9 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:01.534162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2551695a-9de4-45ea-9d20-44a938b4bf23 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:01.538004 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ee128-30d4-4bab-8661-33bf99421311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:01.538617 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ee128-30d4-4bab-8661-33bf99421311 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:01.538913 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ee128-30d4-4bab-8661-33bf99421311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:01.539229 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ee128-30d4-4bab-8661-33bf99421311 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:01.539890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ee128-30d4-4bab-8661-33bf99421311 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:10:01.540211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ee128-30d4-4bab-8661-33bf99421311 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:01.541887 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ee128-30d4-4bab-8661-33bf99421311 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:01.565115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 994/1998] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:10:01.585238 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ee128-30d4-4bab-8661-33bf99421311 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:01.600951 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ee128-30d4-4bab-8661-33bf99421311 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:01.607406 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e2a57-5c8d-40c8-8097-c8fdfee0fc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:01.608247 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e2a57-5c8d-40c8-8097-c8fdfee0fc40 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:01.609287 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e2a57-5c8d-40c8-8097-c8fdfee0fc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:01.609596 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e2a57-5c8d-40c8-8097-c8fdfee0fc40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:01.610401 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e2a57-5c8d-40c8-8097-c8fdfee0fc40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:01.612625 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e2a57-5c8d-40c8-8097-c8fdfee0fc40 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:01.637333 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-2551695a-9de4-45ea-9d20-44a938b4bf23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '127eff42dfb141c3b78ce04636af1f28'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:01.638532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-2551695a-9de4-45ea-9d20-44a938b4bf23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '127eff42dfb141c3b78ce04636af1f28'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:01.649604 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-2551695a-9de4-45ea-9d20-44a938b4bf23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '127eff42dfb141c3b78ce04636af1f28'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:01.664405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-2551695a-9de4-45ea-9d20-44a938b4bf23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '127eff42dfb141c3b78ce04636af1f28'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:01.674263 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-2551695a-9de4-45ea-9d20-44a938b4bf23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '127eff42dfb141c3b78ce04636af1f28'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:01.680144 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 995/1999] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:10:01 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:10:01.684599 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e2a57-5c8d-40c8-8097-c8fdfee0fc40 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:01.686280 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-527e2a57-5c8d-40c8-8097-c8fdfee0fc40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:01.691013 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1005/2000] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:10:01 2022] DELETE /identity/v3/users/127eff42dfb141c3b78ce04636af1f28 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:10:01.803380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e2a57-5c8d-40c8-8097-c8fdfee0fc40 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:01.930763 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91cce4-690e-46a2-9b62-f4c3d3d17f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:01.932165 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91cce4-690e-46a2-9b62-f4c3d3d17f2b admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:01.932923 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91cce4-690e-46a2-9b62-f4c3d3d17f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:01.933403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91cce4-690e-46a2-9b62-f4c3d3d17f2b admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:01.934249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de91cce4-690e-46a2-9b62-f4c3d3d17f2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:01.938044 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de91cce4-690e-46a2-9b62-f4c3d3d17f2b admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:01.978782 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:01.988699 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 996/2001] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:10:01 2022] POST /identity/v3/users => generated 293 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:10:01.993909 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551695a-9de4-45ea-9d20-44a938b4bf23 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:01.994884 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91cce4-690e-46a2-9b62-f4c3d3d17f2b admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:01.995396 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2551695a-9de4-45ea-9d20-44a938b4bf23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:02.010382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91cce4-690e-46a2-9b62-f4c3d3d17f2b admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:02.016074 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551695a-9de4-45ea-9d20-44a938b4bf23 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:02.021578 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d1ac84-df82-4505-8388-488566a70d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:02.022580 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1ac84-df82-4505-8388-488566a70d25 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:02.022580 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1ac84-df82-4505-8388-488566a70d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:02.022865 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1ac84-df82-4505-8388-488566a70d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:02.025135 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d1ac84-df82-4505-8388-488566a70d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:02.026394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb996e97-4b6a-4d23-a593-b7cddea685b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:02.027369 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb996e97-4b6a-4d23-a593-b7cddea685b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:02.028975 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d1ac84-df82-4505-8388-488566a70d25 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:02.029886 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb996e97-4b6a-4d23-a593-b7cddea685b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:02.030174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb996e97-4b6a-4d23-a593-b7cddea685b9 admin admin] PATH_INFO: `/v3/projects/06566ec2efab47ea80f2f10723b478a5` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:02.032819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb996e97-4b6a-4d23-a593-b7cddea685b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06566ec2efab47ea80f2f10723b478a5)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:02.036831 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb996e97-4b6a-4d23-a593-b7cddea685b9 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:02.054104 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 997/2002] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:10:02.058998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d1ac84-df82-4505-8388-488566a70d25 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:02.090536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d1ac84-df82-4505-8388-488566a70d25 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:02.121389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb04b9-7e4b-4e33-ad03-b654d9c10573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:02.122715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb04b9-7e4b-4e33-ad03-b654d9c10573 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:02.123898 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb04b9-7e4b-4e33-ad03-b654d9c10573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:02.124990 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb04b9-7e4b-4e33-ad03-b654d9c10573 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:02.125721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb04b9-7e4b-4e33-ad03-b654d9c10573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:02.128110 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb04b9-7e4b-4e33-ad03-b654d9c10573 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:02.141924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fb996e97-4b6a-4d23-a593-b7cddea685b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06566ec2efab47ea80f2f10723b478a5', 'request_id': 'req-fb996e97-4b6a-4d23-a593-b7cddea685b9'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:02.158000 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 998/2003] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:10:02.159747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb04b9-7e4b-4e33-ad03-b654d9c10573 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:02.174190 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fb996e97-4b6a-4d23-a593-b7cddea685b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06566ec2efab47ea80f2f10723b478a5', 'request_id': 'req-fb996e97-4b6a-4d23-a593-b7cddea685b9'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:02.181863 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb04b9-7e4b-4e33-ad03-b654d9c10573 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:02.183640 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fb996e97-4b6a-4d23-a593-b7cddea685b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06566ec2efab47ea80f2f10723b478a5', 'request_id': 'req-fb996e97-4b6a-4d23-a593-b7cddea685b9'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:02.217211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fb996e97-4b6a-4d23-a593-b7cddea685b9 admin admin] The token cache is being invalidate because project 06566ec2efab47ea80f2f10723b478a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:10:02.217543 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fb996e97-4b6a-4d23-a593-b7cddea685b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:02.218518 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fb996e97-4b6a-4d23-a593-b7cddea685b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:02.223941 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1006/2004] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:10:01 2022] DELETE /identity/v3/projects/06566ec2efab47ea80f2f10723b478a5 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:10:02.226454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b58f46-9a78-4e08-a961-e22ef0f8a7bb None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:02.226903 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b58f46-9a78-4e08-a961-e22ef0f8a7bb None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:02.227197 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b58f46-9a78-4e08-a961-e22ef0f8a7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:02.257908 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-f7b58f46-9a78-4e08-a961-e22ef0f8a7bb None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:02.260341 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f7b58f46-9a78-4e08-a961-e22ef0f8a7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:02.283023 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08389cbf-c9ec-4219-92c8-7c2a9c329b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:02.284898 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08389cbf-c9ec-4219-92c8-7c2a9c329b7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:02.285196 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08389cbf-c9ec-4219-92c8-7c2a9c329b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:02.285603 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08389cbf-c9ec-4219-92c8-7c2a9c329b7f admin admin] PATH_INFO: `/v3/projects/e4cdd66f285b4e848a4ab39484f88243/users/35d5d33be8c943f29ccbfcf6c61163c5/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:02.326296 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08389cbf-c9ec-4219-92c8-7c2a9c329b7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4cdd66f285b4e848a4ab39484f88243, user_id=35d5d33be8c943f29ccbfcf6c61163c5, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:02.330383 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08389cbf-c9ec-4219-92c8-7c2a9c329b7f admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:02.336899 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:02.364816 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 999/2005] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:10:02 2022] PUT /identity/v3/projects/e4cdd66f285b4e848a4ab39484f88243/users/35d5d33be8c943f29ccbfcf6c61163c5/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:10:02.370110 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1007/2006] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:02.373877 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3772dc4-5272-4cf6-b505-03b57e120647 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:02.374544 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3772dc4-5272-4cf6-b505-03b57e120647 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:02.374857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3772dc4-5272-4cf6-b505-03b57e120647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:02.379062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc18e1d-98b9-4b65-8e17-8756321de944 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:02.381095 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc18e1d-98b9-4b65-8e17-8756321de944 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:02.381902 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc18e1d-98b9-4b65-8e17-8756321de944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:02.424442 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-d3772dc4-5272-4cf6-b505-03b57e120647 None None] MFA Rules not processed for user `35d5d33be8c943f29ccbfcf6c61163c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:02.425998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d3772dc4-5272-4cf6-b505-03b57e120647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:02.440980 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-6dc18e1d-98b9-4b65-8e17-8756321de944 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:02.444110 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-6dc18e1d-98b9-4b65-8e17-8756321de944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:02.532619 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1008/2007] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:10:02.536758 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc18e1d-98b9-4b65-8e17-8756321de944 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:02.544728 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1000/2008] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:02.563129 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3772dc4-5272-4cf6-b505-03b57e120647 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:02.564966 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d3772dc4-5272-4cf6-b505-03b57e120647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:02.567568 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc18e1d-98b9-4b65-8e17-8756321de944 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:02.582424 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9acef-efe7-4b5a-aa11-2e207cfaa25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:02.583969 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9acef-efe7-4b5a-aa11-2e207cfaa25b admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:02.584323 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9acef-efe7-4b5a-aa11-2e207cfaa25b admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:02.585804 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9acef-efe7-4b5a-aa11-2e207cfaa25b admin admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:02.586571 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9acef-efe7-4b5a-aa11-2e207cfaa25b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:10:02.587750 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9acef-efe7-4b5a-aa11-2e207cfaa25b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:02.604440 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9acef-efe7-4b5a-aa11-2e207cfaa25b admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:02.628953 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1009/2009] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:10:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:10:02.630072 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9acef-efe7-4b5a-aa11-2e207cfaa25b admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:02.650950 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9acef-efe7-4b5a-aa11-2e207cfaa25b admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:02.659210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e136715-2a82-4b0d-afc2-13772e5a4a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:02.660609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e136715-2a82-4b0d-afc2-13772e5a4a11 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:02.661632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e136715-2a82-4b0d-afc2-13772e5a4a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:02.665496 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e136715-2a82-4b0d-afc2-13772e5a4a11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:02.667932 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e136715-2a82-4b0d-afc2-13772e5a4a11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:02.672874 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e136715-2a82-4b0d-afc2-13772e5a4a11 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:02.695851 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3772dc4-5272-4cf6-b505-03b57e120647 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:02.699677 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f1a9c-dfb3-4839-90c0-d1868f5d1472 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:02.700355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f1a9c-dfb3-4839-90c0-d1868f5d1472 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:02.700694 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f1a9c-dfb3-4839-90c0-d1868f5d1472 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:02.700976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f1a9c-dfb3-4839-90c0-d1868f5d1472 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:02.724189 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:02.733500 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f1a9c-dfb3-4839-90c0-d1868f5d1472 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:02.736675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f1a9c-dfb3-4839-90c0-d1868f5d1472 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:02.766363 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1010/2010] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:10:02 2022] POST /identity/v3/projects => generated 377 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:10:02.771601 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e136715-2a82-4b0d-afc2-13772e5a4a11 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:02.787493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1001/2011] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:10:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:10:02.862905 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e136715-2a82-4b0d-afc2-13772e5a4a11 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:02.867930 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e8652-5ca6-4395-b763-322730d34815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:02.868800 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e8652-5ca6-4395-b763-322730d34815 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:02.869251 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e8652-5ca6-4395-b763-322730d34815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:02.869648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e8652-5ca6-4395-b763-322730d34815 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:02.870368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e8652-5ca6-4395-b763-322730d34815 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:02.873568 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e8652-5ca6-4395-b763-322730d34815 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:02.904246 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1011/2012] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:10:02 2022] POST /identity/v3/users => generated 289 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:10:02.907985 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f1a9c-dfb3-4839-90c0-d1868f5d1472 service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:02.923320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f1a9c-dfb3-4839-90c0-d1868f5d1472 service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:02.931381 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65288609-601f-411b-ae78-db654f01ab01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:02.932589 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65288609-601f-411b-ae78-db654f01ab01 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:02.933079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65288609-601f-411b-ae78-db654f01ab01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:02.933491 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65288609-601f-411b-ae78-db654f01ab01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:02.934195 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65288609-601f-411b-ae78-db654f01ab01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:02.936363 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65288609-601f-411b-ae78-db654f01ab01 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:02.947459 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1002/2013] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:10:02.953968 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e8652-5ca6-4395-b763-322730d34815 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:02.965541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e8652-5ca6-4395-b763-322730d34815 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:02.971712 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218146ab-32a7-48f8-b3e1-bb2b7ba40327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:02.972436 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218146ab-32a7-48f8-b3e1-bb2b7ba40327 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:02.972843 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218146ab-32a7-48f8-b3e1-bb2b7ba40327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:02.973191 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218146ab-32a7-48f8-b3e1-bb2b7ba40327 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:02.973977 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218146ab-32a7-48f8-b3e1-bb2b7ba40327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:02.975751 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218146ab-32a7-48f8-b3e1-bb2b7ba40327 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:02.993084 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1012/2014] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:10:02.994499 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65288609-601f-411b-ae78-db654f01ab01 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:03.010464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65288609-601f-411b-ae78-db654f01ab01 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:03.020287 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba370b4b-111a-4402-8156-5b0cd3c89e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:03.021142 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba370b4b-111a-4402-8156-5b0cd3c89e63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:03.021433 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba370b4b-111a-4402-8156-5b0cd3c89e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:03.021733 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba370b4b-111a-4402-8156-5b0cd3c89e63 admin admin] PATH_INFO: `/v3/projects/af0eff3dcaa447a6ad585ffc72f64d78/users/f0cf683aa4d14dea974f0febffbbeacc/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:03.056830 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba370b4b-111a-4402-8156-5b0cd3c89e63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af0eff3dcaa447a6ad585ffc72f64d78, user_id=f0cf683aa4d14dea974f0febffbbeacc, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:03.059194 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba370b4b-111a-4402-8156-5b0cd3c89e63 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:03.101487 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1003/2015] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:10:02 2022] PUT /identity/v3/projects/af0eff3dcaa447a6ad585ffc72f64d78/users/f0cf683aa4d14dea974f0febffbbeacc/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:10:03.107917 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774b431-3be9-4a8f-a740-dcadccd2b46d None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:03.108246 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774b431-3be9-4a8f-a740-dcadccd2b46d None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:03.110450 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774b431-3be9-4a8f-a740-dcadccd2b46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:03.150963 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-6774b431-3be9-4a8f-a740-dcadccd2b46d None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:03.152620 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-6774b431-3be9-4a8f-a740-dcadccd2b46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:03.246630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:03.315522 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1013/2016] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:03.331347 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba370b4b-111a-4402-8156-5b0cd3c89e63 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:03.335855 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ba370b4b-111a-4402-8156-5b0cd3c89e63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:03.463610 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba370b4b-111a-4402-8156-5b0cd3c89e63 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:03.468870 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71b007-8c6e-402b-88d9-cac2d196bb87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:03.470298 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71b007-8c6e-402b-88d9-cac2d196bb87 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:03.472948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71b007-8c6e-402b-88d9-cac2d196bb87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:03.472948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71b007-8c6e-402b-88d9-cac2d196bb87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:03.489871 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71b007-8c6e-402b-88d9-cac2d196bb87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:03.492091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71b007-8c6e-402b-88d9-cac2d196bb87 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:03.543186 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1004/2017] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:10:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:10:05.197491 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774b431-3be9-4a8f-a740-dcadccd2b46d None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:05.199701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-6774b431-3be9-4a8f-a740-dcadccd2b46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:05.221192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774b431-3be9-4a8f-a740-dcadccd2b46d None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:05.228986 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da43294-a3b9-44f0-886e-941672e27900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:05.230670 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da43294-a3b9-44f0-886e-941672e27900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:05.231194 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da43294-a3b9-44f0-886e-941672e27900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:05.231732 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da43294-a3b9-44f0-886e-941672e27900 admin admin] PATH_INFO: `/v3/users/e1dc2375d02249eb93623f3484585273` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:05.234795 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da43294-a3b9-44f0-886e-941672e27900 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1dc2375d02249eb93623f3484585273)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:05.238043 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da43294-a3b9-44f0-886e-941672e27900 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:05.359723 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-2da43294-a3b9-44f0-886e-941672e27900 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1dc2375d02249eb93623f3484585273'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:05.362542 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-2da43294-a3b9-44f0-886e-941672e27900 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1dc2375d02249eb93623f3484585273'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:05.392037 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-2da43294-a3b9-44f0-886e-941672e27900 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1dc2375d02249eb93623f3484585273'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:05.476730 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-2da43294-a3b9-44f0-886e-941672e27900 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1dc2375d02249eb93623f3484585273'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:05.503804 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-2da43294-a3b9-44f0-886e-941672e27900 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1dc2375d02249eb93623f3484585273'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:05.521069 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1014/2018] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:10:05 2022] DELETE /identity/v3/users/e1dc2375d02249eb93623f3484585273 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:10:05.523709 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e1859-88e2-45d8-a1cb-ffe3696179de None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:05.524182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e1859-88e2-45d8-a1cb-ffe3696179de None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:05.524300 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e1859-88e2-45d8-a1cb-ffe3696179de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:05.565590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-dc9e1859-88e2-45d8-a1cb-ffe3696179de None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:05.567005 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-dc9e1859-88e2-45d8-a1cb-ffe3696179de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:05.695149 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1005/2019] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:05.711179 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da43294-a3b9-44f0-886e-941672e27900 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:05.712869 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2da43294-a3b9-44f0-886e-941672e27900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:05.873920 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da43294-a3b9-44f0-886e-941672e27900 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:05.883859 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717e2e9-1ef7-4b7f-8d83-3e429e1dfa18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:05.884734 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717e2e9-1ef7-4b7f-8d83-3e429e1dfa18 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:05.885226 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717e2e9-1ef7-4b7f-8d83-3e429e1dfa18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:05.885389 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717e2e9-1ef7-4b7f-8d83-3e429e1dfa18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:05.910787 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f717e2e9-1ef7-4b7f-8d83-3e429e1dfa18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:05.912407 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f717e2e9-1ef7-4b7f-8d83-3e429e1dfa18 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:05.928869 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1015/2020] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:10:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:10:06.880729 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e1859-88e2-45d8-a1cb-ffe3696179de None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:06.882301 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-dc9e1859-88e2-45d8-a1cb-ffe3696179de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:06.913110 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e1859-88e2-45d8-a1cb-ffe3696179de None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:06.919678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717e2e9-1ef7-4b7f-8d83-3e429e1dfa18 service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:06.922323 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f717e2e9-1ef7-4b7f-8d83-3e429e1dfa18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:06.931447 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f6b56-a22d-4265-b901-69d6ac977d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:06.932125 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f6b56-a22d-4265-b901-69d6ac977d4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:06.932412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f6b56-a22d-4265-b901-69d6ac977d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:06.932717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f6b56-a22d-4265-b901-69d6ac977d4c admin admin] PATH_INFO: `/v3/users/35d5d33be8c943f29ccbfcf6c61163c5` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:06.935333 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20f6b56-a22d-4265-b901-69d6ac977d4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=35d5d33be8c943f29ccbfcf6c61163c5)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:06.937395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20f6b56-a22d-4265-b901-69d6ac977d4c admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:06.959630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717e2e9-1ef7-4b7f-8d83-3e429e1dfa18 service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:06.986908 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2bbcc7-d49c-40f4-b212-f7ccc37e4147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:06.987854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2bbcc7-d49c-40f4-b212-f7ccc37e4147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:06.987854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2bbcc7-d49c-40f4-b212-f7ccc37e4147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:06.987854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2bbcc7-d49c-40f4-b212-f7ccc37e4147 admin admin] PATH_INFO: `/v3/projects/654274e1a7584c6aabf1e1fa44223f21` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:06.989067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2bbcc7-d49c-40f4-b212-f7ccc37e4147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=654274e1a7584c6aabf1e1fa44223f21)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:06.991304 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2bbcc7-d49c-40f4-b212-f7ccc37e4147 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:07.096700 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-b20f6b56-a22d-4265-b901-69d6ac977d4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35d5d33be8c943f29ccbfcf6c61163c5'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:07.110649 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-b20f6b56-a22d-4265-b901-69d6ac977d4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35d5d33be8c943f29ccbfcf6c61163c5'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:07.112387 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:07.141097 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:07.145998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-6a2bbcc7-d49c-40f4-b212-f7ccc37e4147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '654274e1a7584c6aabf1e1fa44223f21', 'request_id': 'req-6a2bbcc7-d49c-40f4-b212-f7ccc37e4147'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:07.146973 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-b20f6b56-a22d-4265-b901-69d6ac977d4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35d5d33be8c943f29ccbfcf6c61163c5'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:07.147876 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-b20f6b56-a22d-4265-b901-69d6ac977d4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35d5d33be8c943f29ccbfcf6c61163c5'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:07.148987 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-b20f6b56-a22d-4265-b901-69d6ac977d4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35d5d33be8c943f29ccbfcf6c61163c5'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:07.165425 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-6a2bbcc7-d49c-40f4-b212-f7ccc37e4147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '654274e1a7584c6aabf1e1fa44223f21', 'request_id': 'req-6a2bbcc7-d49c-40f4-b212-f7ccc37e4147'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:07.179807 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1006/2021] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:10:06 2022] DELETE /identity/v3/users/35d5d33be8c943f29ccbfcf6c61163c5 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:10:07.196702 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-6a2bbcc7-d49c-40f4-b212-f7ccc37e4147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '654274e1a7584c6aabf1e1fa44223f21', 'request_id': 'req-6a2bbcc7-d49c-40f4-b212-f7ccc37e4147'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:07.196958 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431a82b-daaf-4547-983c-bf4a9c5af72b None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:07.196958 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431a82b-daaf-4547-983c-bf4a9c5af72b None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:07.196958 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431a82b-daaf-4547-983c-bf4a9c5af72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:07.215945 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-6a2bbcc7-d49c-40f4-b212-f7ccc37e4147 admin admin] The token cache is being invalidate because project 654274e1a7584c6aabf1e1fa44223f21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:10:07.216248 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-6a2bbcc7-d49c-40f4-b212-f7ccc37e4147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:07.217329 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-6a2bbcc7-d49c-40f4-b212-f7ccc37e4147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:07.221281 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1016/2022] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:10:06 2022] DELETE /identity/v3/projects/654274e1a7584c6aabf1e1fa44223f21 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:10:07.225057 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2bbcc7-d49c-40f4-b212-f7ccc37e4147 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:07.228351 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-6a2bbcc7-d49c-40f4-b212-f7ccc37e4147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:07.234675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-5431a82b-daaf-4547-983c-bf4a9c5af72b None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:07.235904 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5431a82b-daaf-4547-983c-bf4a9c5af72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:07.400942 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2bbcc7-d49c-40f4-b212-f7ccc37e4147 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:07.410070 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a35506-15df-491e-9205-d9a3a20bd5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:07.410930 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a35506-15df-491e-9205-d9a3a20bd5b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:07.411370 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a35506-15df-491e-9205-d9a3a20bd5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:07.411694 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a35506-15df-491e-9205-d9a3a20bd5b9 admin admin] PATH_INFO: `/v3/users/b59a000682b44dce878eb7f775d5e558` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:07.413420 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a35506-15df-491e-9205-d9a3a20bd5b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b59a000682b44dce878eb7f775d5e558)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:07.416344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a35506-15df-491e-9205-d9a3a20bd5b9 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:07.417049 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1007/2023] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:07.476630 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431a82b-daaf-4547-983c-bf4a9c5af72b None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:07.478408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5431a82b-daaf-4547-983c-bf4a9c5af72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:07.540911 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-33a35506-15df-491e-9205-d9a3a20bd5b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b59a000682b44dce878eb7f775d5e558'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:07.541579 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-33a35506-15df-491e-9205-d9a3a20bd5b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b59a000682b44dce878eb7f775d5e558'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:07.562924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-33a35506-15df-491e-9205-d9a3a20bd5b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b59a000682b44dce878eb7f775d5e558'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:07.589187 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431a82b-daaf-4547-983c-bf4a9c5af72b None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:07.594533 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-33a35506-15df-491e-9205-d9a3a20bd5b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b59a000682b44dce878eb7f775d5e558'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:07.596833 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff3bce-d3b6-4e1e-b102-7a7a4c19c23a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:07.601247 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff3bce-d3b6-4e1e-b102-7a7a4c19c23a service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:07.605145 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff3bce-d3b6-4e1e-b102-7a7a4c19c23a service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:07.606558 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dff3bce-d3b6-4e1e-b102-7a7a4c19c23a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:07.609801 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-33a35506-15df-491e-9205-d9a3a20bd5b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b59a000682b44dce878eb7f775d5e558'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:07.613996 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-4dff3bce-d3b6-4e1e-b102-7a7a4c19c23a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:07.632262 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1017/2024] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:10:07 2022] DELETE /identity/v3/users/b59a000682b44dce878eb7f775d5e558 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:10:07.644896 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff3bce-d3b6-4e1e-b102-7a7a4c19c23a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:07.646479 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dff3bce-d3b6-4e1e-b102-7a7a4c19c23a service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:07.675720 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1008/2025] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:10:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:10:09.170371 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9711dd-0021-46b4-aaa5-99a6128b6fdc None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:09.170778 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9711dd-0021-46b4-aaa5-99a6128b6fdc None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:09.170951 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9711dd-0021-46b4-aaa5-99a6128b6fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:09.246093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-cc9711dd-0021-46b4-aaa5-99a6128b6fdc None None] MFA Rules not processed for user `fb594a081d194e709023dc7c557afe4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:09.248613 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-cc9711dd-0021-46b4-aaa5-99a6128b6fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:09.317907 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:09.357663 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1018/2026] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:09.363457 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff3bce-d3b6-4e1e-b102-7a7a4c19c23a service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:09.371128 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-4dff3bce-d3b6-4e1e-b102-7a7a4c19c23a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:09.385569 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dff3bce-d3b6-4e1e-b102-7a7a4c19c23a service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:09.514357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a0842-c30a-4b37-b8ac-80f4690679f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:09.517058 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a0842-c30a-4b37-b8ac-80f4690679f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:09.517770 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a0842-c30a-4b37-b8ac-80f4690679f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:09.518160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a0842-c30a-4b37-b8ac-80f4690679f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:09.519167 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a0842-c30a-4b37-b8ac-80f4690679f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:09.521334 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a0842-c30a-4b37-b8ac-80f4690679f2 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:09.576159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1009/2027] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:10:09 2022] POST /identity/v3/projects => generated 385 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:10:09.582075 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9711dd-0021-46b4-aaa5-99a6128b6fdc None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:09.593855 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a0842-c30a-4b37-b8ac-80f4690679f2 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:09.601975 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-733a0842-c30a-4b37-b8ac-80f4690679f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:09.606496 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9711dd-0021-46b4-aaa5-99a6128b6fdc None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:09.608078 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:09.708764 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea8b4de-338e-4429-9874-39c0665feed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:09.710819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8b4de-338e-4429-9874-39c0665feed8 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:09.711847 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8b4de-338e-4429-9874-39c0665feed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:09.712928 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8b4de-338e-4429-9874-39c0665feed8 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:09.715359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea8b4de-338e-4429-9874-39c0665feed8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:09.721598 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea8b4de-338e-4429-9874-39c0665feed8 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:09.741611 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a0842-c30a-4b37-b8ac-80f4690679f2 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:09.749509 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0960537-8b8c-48d1-8b71-ed3e104db88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:09.751283 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0960537-8b8c-48d1-8b71-ed3e104db88c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:09.752407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0960537-8b8c-48d1-8b71-ed3e104db88c admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:09.752902 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0960537-8b8c-48d1-8b71-ed3e104db88c admin admin] PATH_INFO: `/v3/projects/34c2ccc2b2704a47be0f563845e2721b` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:09.754225 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0960537-8b8c-48d1-8b71-ed3e104db88c admin admin] RBAC: Authorizing `identity:delete_project(project_id=34c2ccc2b2704a47be0f563845e2721b)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:09.756368 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0960537-8b8c-48d1-8b71-ed3e104db88c admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:09.768670 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1019/2028] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:10:09 2022] POST /identity/v3/users => generated 299 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:10:09.772756 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea8b4de-338e-4429-9874-39c0665feed8 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:09.795757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea8b4de-338e-4429-9874-39c0665feed8 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:09.809239 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8ae30-9801-46da-af48-35b819d2f0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:09.809962 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8ae30-9801-46da-af48-35b819d2f0ba admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:09.810282 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8ae30-9801-46da-af48-35b819d2f0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:09.810565 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8ae30-9801-46da-af48-35b819d2f0ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:09.811221 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8ae30-9801-46da-af48-35b819d2f0ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:09.815287 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8ae30-9801-46da-af48-35b819d2f0ba admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:09.848528 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1020/2029] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:10:09.851685 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8ae30-9801-46da-af48-35b819d2f0ba admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:09.876384 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8ae30-9801-46da-af48-35b819d2f0ba admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:09.900260 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d0960537-8b8c-48d1-8b71-ed3e104db88c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34c2ccc2b2704a47be0f563845e2721b', 'request_id': 'req-d0960537-8b8c-48d1-8b71-ed3e104db88c'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:09.910222 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d0960537-8b8c-48d1-8b71-ed3e104db88c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34c2ccc2b2704a47be0f563845e2721b', 'request_id': 'req-d0960537-8b8c-48d1-8b71-ed3e104db88c'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:09.930355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d0960537-8b8c-48d1-8b71-ed3e104db88c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34c2ccc2b2704a47be0f563845e2721b', 'request_id': 'req-d0960537-8b8c-48d1-8b71-ed3e104db88c'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:09.950099 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d0960537-8b8c-48d1-8b71-ed3e104db88c admin admin] The token cache is being invalidate because project 34c2ccc2b2704a47be0f563845e2721b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:10:09.950703 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d0960537-8b8c-48d1-8b71-ed3e104db88c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:09.952004 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d0960537-8b8c-48d1-8b71-ed3e104db88c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:09.955833 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1010/2030] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:10:09 2022] DELETE /identity/v3/projects/34c2ccc2b2704a47be0f563845e2721b => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:10:09.960049 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0960537-8b8c-48d1-8b71-ed3e104db88c admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:09.961653 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d0960537-8b8c-48d1-8b71-ed3e104db88c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:10.066884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bb06b-4d26-42cd-b8fb-6b93f07c2eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:10.067665 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bb06b-4d26-42cd-b8fb-6b93f07c2eba admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:10.068064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bb06b-4d26-42cd-b8fb-6b93f07c2eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:10.068411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bb06b-4d26-42cd-b8fb-6b93f07c2eba admin admin] PATH_INFO: `/v3/projects/93b00f1b589e4be8b09523fa4cafd09b/users/223a5f4ab8914d1c83410fdb2e64390b/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:10.111917 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2bb06b-4d26-42cd-b8fb-6b93f07c2eba admin admin] RBAC: Authorizing `identity:create_grant(project_id=93b00f1b589e4be8b09523fa4cafd09b, user_id=223a5f4ab8914d1c83410fdb2e64390b, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:10.114862 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0960537-8b8c-48d1-8b71-ed3e104db88c admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:10.121361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b97a7f-74c5-4269-a042-70d20a799c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:10.122230 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b97a7f-74c5-4269-a042-70d20a799c60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:10.122683 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b97a7f-74c5-4269-a042-70d20a799c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:10.123367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b97a7f-74c5-4269-a042-70d20a799c60 admin admin] PATH_INFO: `/v3/users/ff6311f00dab4782b616598dbc99503f` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:10.126496 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b97a7f-74c5-4269-a042-70d20a799c60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff6311f00dab4782b616598dbc99503f)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:10.128121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2bb06b-4d26-42cd-b8fb-6b93f07c2eba admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:10.128848 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b97a7f-74c5-4269-a042-70d20a799c60 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:10.180299 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1021/2031] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:10:09 2022] PUT /identity/v3/projects/93b00f1b589e4be8b09523fa4cafd09b/users/223a5f4ab8914d1c83410fdb2e64390b/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:10:10.183656 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bb06b-4d26-42cd-b8fb-6b93f07c2eba admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:10.185600 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8c2bb06b-4d26-42cd-b8fb-6b93f07c2eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:10.228975 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:10.242481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-86b97a7f-74c5-4269-a042-70d20a799c60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff6311f00dab4782b616598dbc99503f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:10.258853 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-86b97a7f-74c5-4269-a042-70d20a799c60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff6311f00dab4782b616598dbc99503f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:10.288862 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-86b97a7f-74c5-4269-a042-70d20a799c60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff6311f00dab4782b616598dbc99503f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:10.289603 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-86b97a7f-74c5-4269-a042-70d20a799c60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff6311f00dab4782b616598dbc99503f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:10.293910 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-86b97a7f-74c5-4269-a042-70d20a799c60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff6311f00dab4782b616598dbc99503f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:10.317925 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:10.324279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1011/2032] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:10:09 2022] DELETE /identity/v3/users/ff6311f00dab4782b616598dbc99503f => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:10:10.330177 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b97a7f-74c5-4269-a042-70d20a799c60 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:10.332476 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-86b97a7f-74c5-4269-a042-70d20a799c60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:10.373006 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bb06b-4d26-42cd-b8fb-6b93f07c2eba admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:10.506138 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b97a7f-74c5-4269-a042-70d20a799c60 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:10.518364 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926a5ad-266a-4087-b48f-b38ec0bdca6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:10.518911 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926a5ad-266a-4087-b48f-b38ec0bdca6f admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:10.519195 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926a5ad-266a-4087-b48f-b38ec0bdca6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:10.519472 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926a5ad-266a-4087-b48f-b38ec0bdca6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:10.520118 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926a5ad-266a-4087-b48f-b38ec0bdca6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:10.522307 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926a5ad-266a-4087-b48f-b38ec0bdca6f admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:10.527022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12049a7-56e5-411e-a10e-769ecbfb577d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:10.527672 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12049a7-56e5-411e-a10e-769ecbfb577d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:10.528591 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12049a7-56e5-411e-a10e-769ecbfb577d admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:10.528878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12049a7-56e5-411e-a10e-769ecbfb577d admin admin] PATH_INFO: `/v3/projects/e4cdd66f285b4e848a4ab39484f88243` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:10.532158 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12049a7-56e5-411e-a10e-769ecbfb577d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4cdd66f285b4e848a4ab39484f88243)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:10.539519 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12049a7-56e5-411e-a10e-769ecbfb577d admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:10.555711 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1012/2033] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:10:10.559068 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926a5ad-266a-4087-b48f-b38ec0bdca6f admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:10.579991 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926a5ad-266a-4087-b48f-b38ec0bdca6f admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:10.620278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece529fc-9d66-4bd8-8367-39ae62110a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:10.621008 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece529fc-9d66-4bd8-8367-39ae62110a19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:10.622237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece529fc-9d66-4bd8-8367-39ae62110a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:10.622524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece529fc-9d66-4bd8-8367-39ae62110a19 admin admin] PATH_INFO: `/v3/projects/93b00f1b589e4be8b09523fa4cafd09b/users/223a5f4ab8914d1c83410fdb2e64390b/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:10.638627 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece529fc-9d66-4bd8-8367-39ae62110a19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93b00f1b589e4be8b09523fa4cafd09b, user_id=223a5f4ab8914d1c83410fdb2e64390b, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:10.657615 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece529fc-9d66-4bd8-8367-39ae62110a19 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:10.711138 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1013/2034] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:10:10 2022] PUT /identity/v3/projects/93b00f1b589e4be8b09523fa4cafd09b/users/223a5f4ab8914d1c83410fdb2e64390b/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:10:10.723179 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b12049a7-56e5-411e-a10e-769ecbfb577d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4cdd66f285b4e848a4ab39484f88243', 'request_id': 'req-b12049a7-56e5-411e-a10e-769ecbfb577d'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:10.745430 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b12049a7-56e5-411e-a10e-769ecbfb577d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4cdd66f285b4e848a4ab39484f88243', 'request_id': 'req-b12049a7-56e5-411e-a10e-769ecbfb577d'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:10.764216 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b12049a7-56e5-411e-a10e-769ecbfb577d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4cdd66f285b4e848a4ab39484f88243', 'request_id': 'req-b12049a7-56e5-411e-a10e-769ecbfb577d'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:10.789943 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b12049a7-56e5-411e-a10e-769ecbfb577d admin admin] The token cache is being invalidate because project e4cdd66f285b4e848a4ab39484f88243 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:10:10.790234 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b12049a7-56e5-411e-a10e-769ecbfb577d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:10.791099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b12049a7-56e5-411e-a10e-769ecbfb577d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:10.793648 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1022/2035] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:10:10 2022] DELETE /identity/v3/projects/e4cdd66f285b4e848a4ab39484f88243 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:10:10.800935 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbea5a-7106-44a7-990c-2310b7354828 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:10.801910 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbea5a-7106-44a7-990c-2310b7354828 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:10.803847 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbea5a-7106-44a7-990c-2310b7354828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:10.837211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12049a7-56e5-411e-a10e-769ecbfb577d admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:10.838666 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b12049a7-56e5-411e-a10e-769ecbfb577d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:10.839508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-9ecbea5a-7106-44a7-990c-2310b7354828 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:10.840733 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9ecbea5a-7106-44a7-990c-2310b7354828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:10.959765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1014/2036] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:10.970417 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cbdf3-c2f7-4f8a-97ee-54a3819e8693 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:10.971038 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cbdf3-c2f7-4f8a-97ee-54a3819e8693 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:10.971611 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cbdf3-c2f7-4f8a-97ee-54a3819e8693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:11.022355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12049a7-56e5-411e-a10e-769ecbfb577d admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:11.030236 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-0c5cbdf3-c2f7-4f8a-97ee-54a3819e8693 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:11.034851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6fd241-4f83-47c8-9ab2-073035e8ad03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:11.035433 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-0c5cbdf3-c2f7-4f8a-97ee-54a3819e8693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:11.039106 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6fd241-4f83-47c8-9ab2-073035e8ad03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:11.039513 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6fd241-4f83-47c8-9ab2-073035e8ad03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:11.039809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6fd241-4f83-47c8-9ab2-073035e8ad03 admin admin] PATH_INFO: `/v3/projects/552c0264796e42e9a9a5af5dc288d1b1` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:11.042163 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6fd241-4f83-47c8-9ab2-073035e8ad03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=552c0264796e42e9a9a5af5dc288d1b1)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:11.051960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6fd241-4f83-47c8-9ab2-073035e8ad03 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:11.055838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1015/2037] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:11.062900 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cbdf3-c2f7-4f8a-97ee-54a3819e8693 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:11.079765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cbdf3-c2f7-4f8a-97ee-54a3819e8693 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:11.096832 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656318d-f134-45bf-baa9-b119e6568ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:11.097520 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656318d-f134-45bf-baa9-b119e6568ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:11.098003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656318d-f134-45bf-baa9-b119e6568ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:11.098121 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656318d-f134-45bf-baa9-b119e6568ee0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:11.098728 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656318d-f134-45bf-baa9-b119e6568ee0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:10:11.099133 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656318d-f134-45bf-baa9-b119e6568ee0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:11.100821 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656318d-f134-45bf-baa9-b119e6568ee0 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:11.125143 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:11.132369 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1016/2038] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:10:11.136093 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656318d-f134-45bf-baa9-b119e6568ee0 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:11.151319 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656318d-f134-45bf-baa9-b119e6568ee0 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:11.169314 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:11.200117 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5c6fd241-4f83-47c8-9ab2-073035e8ad03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '552c0264796e42e9a9a5af5dc288d1b1', 'request_id': 'req-5c6fd241-4f83-47c8-9ab2-073035e8ad03'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:11.233362 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5c6fd241-4f83-47c8-9ab2-073035e8ad03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '552c0264796e42e9a9a5af5dc288d1b1', 'request_id': 'req-5c6fd241-4f83-47c8-9ab2-073035e8ad03'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:11.287211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5c6fd241-4f83-47c8-9ab2-073035e8ad03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '552c0264796e42e9a9a5af5dc288d1b1', 'request_id': 'req-5c6fd241-4f83-47c8-9ab2-073035e8ad03'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:11.293709 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd01490-f8ec-485f-b7bc-05b916368ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:11.294198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd01490-f8ec-485f-b7bc-05b916368ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:11.294536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd01490-f8ec-485f-b7bc-05b916368ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:11.294820 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd01490-f8ec-485f-b7bc-05b916368ff0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:11.295616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd01490-f8ec-485f-b7bc-05b916368ff0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:11.301763 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd01490-f8ec-485f-b7bc-05b916368ff0 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:11.339627 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5c6fd241-4f83-47c8-9ab2-073035e8ad03 admin admin] The token cache is being invalidate because project 552c0264796e42e9a9a5af5dc288d1b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:10:11.340464 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5c6fd241-4f83-47c8-9ab2-073035e8ad03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:11.347208 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5c6fd241-4f83-47c8-9ab2-073035e8ad03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:11.351810 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1023/2039] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:10:10 2022] DELETE /identity/v3/projects/552c0264796e42e9a9a5af5dc288d1b1 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:10:11.359097 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60759ba7-179a-4aac-81a9-e3d10f224005 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:11.359493 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60759ba7-179a-4aac-81a9-e3d10f224005 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:11.359676 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60759ba7-179a-4aac-81a9-e3d10f224005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:11.389874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1017/2040] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:10:11 2022] POST /identity/v3/projects => generated 371 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:10:11.392514 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd01490-f8ec-485f-b7bc-05b916368ff0 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:11.395859 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-cdd01490-f8ec-485f-b7bc-05b916368ff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:11.486607 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-60759ba7-179a-4aac-81a9-e3d10f224005 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:11.491345 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-60759ba7-179a-4aac-81a9-e3d10f224005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:11.642923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1024/2041] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:11.646975 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd01490-f8ec-485f-b7bc-05b916368ff0 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:11.657139 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a49ac4-6132-4bc5-9d01-18807ef19cda None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:11.658428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a49ac4-6132-4bc5-9d01-18807ef19cda None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:11.658936 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a49ac4-6132-4bc5-9d01-18807ef19cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:11.659847 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d03ba5-ff8f-4f31-825b-d906db878447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:11.665523 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d03ba5-ff8f-4f31-825b-d906db878447 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:11.665681 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d03ba5-ff8f-4f31-825b-d906db878447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:11.665909 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d03ba5-ff8f-4f31-825b-d906db878447 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:11.666578 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d03ba5-ff8f-4f31-825b-d906db878447 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:11.668616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d03ba5-ff8f-4f31-825b-d906db878447 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:11.710282 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1018/2042] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:10:11 2022] POST /identity/v3/users => generated 286 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:10:11.714408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d03ba5-ff8f-4f31-825b-d906db878447 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:11.746048 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d03ba5-ff8f-4f31-825b-d906db878447 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:11.748336 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-99a49ac4-6132-4bc5-9d01-18807ef19cda None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:11.752364 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d82dc-717d-47a9-b918-0542d6ca285b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:11.753394 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d82dc-717d-47a9-b918-0542d6ca285b admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:11.753717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d82dc-717d-47a9-b918-0542d6ca285b admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:11.754035 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d82dc-717d-47a9-b918-0542d6ca285b admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:11.754766 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d82dc-717d-47a9-b918-0542d6ca285b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:11.755709 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-99a49ac4-6132-4bc5-9d01-18807ef19cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:11.756792 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d82dc-717d-47a9-b918-0542d6ca285b admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:11.786857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1019/2043] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:10:11.787846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d82dc-717d-47a9-b918-0542d6ca285b admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:11.810638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1025/2044] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:11.812429 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a49ac4-6132-4bc5-9d01-18807ef19cda None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:11.821736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d82dc-717d-47a9-b918-0542d6ca285b admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:11.872914 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7344411-a1bd-4f5b-a3b8-0bb873f37765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:11.873602 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7344411-a1bd-4f5b-a3b8-0bb873f37765 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:11.873904 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7344411-a1bd-4f5b-a3b8-0bb873f37765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:11.874281 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7344411-a1bd-4f5b-a3b8-0bb873f37765 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:11.876578 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7344411-a1bd-4f5b-a3b8-0bb873f37765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:11.878220 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7344411-a1bd-4f5b-a3b8-0bb873f37765 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:11.883264 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a49ac4-6132-4bc5-9d01-18807ef19cda None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:11.913101 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6bf0c-6102-4e75-9ce1-b79a1df4d2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:11.916673 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6bf0c-6102-4e75-9ce1-b79a1df4d2fc admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:11.919950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1020/2045] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:10:11.924538 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7344411-a1bd-4f5b-a3b8-0bb873f37765 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:11.924957 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6bf0c-6102-4e75-9ce1-b79a1df4d2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:11.927201 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6bf0c-6102-4e75-9ce1-b79a1df4d2fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:11.928935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6bf0c-6102-4e75-9ce1-b79a1df4d2fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:10:11.932866 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6bf0c-6102-4e75-9ce1-b79a1df4d2fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:11.938140 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6bf0c-6102-4e75-9ce1-b79a1df4d2fc admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:11.964279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7344411-a1bd-4f5b-a3b8-0bb873f37765 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:11.988789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67892a35-5551-4f70-b7ab-be98d7cff92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:11.991027 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67892a35-5551-4f70-b7ab-be98d7cff92b admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:11.992740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67892a35-5551-4f70-b7ab-be98d7cff92b admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:11.994694 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67892a35-5551-4f70-b7ab-be98d7cff92b admin admin] PATH_INFO: `/v3/projects/50b9a2f218ee4917b4f6539c345b0eaf/users/52dd55cd6b724e478bbe179f0d1f5f4a/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:11.996865 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1026/2046] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:10:12.002622 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6bf0c-6102-4e75-9ce1-b79a1df4d2fc admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:12.055921 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6bf0c-6102-4e75-9ce1-b79a1df4d2fc admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:12.083245 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67892a35-5551-4f70-b7ab-be98d7cff92b admin admin] RBAC: Authorizing `identity:create_grant(project_id=50b9a2f218ee4917b4f6539c345b0eaf, user_id=52dd55cd6b724e478bbe179f0d1f5f4a, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:12.091198 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f31514-3f82-4d44-868d-6e9fc38219dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:12.091578 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67892a35-5551-4f70-b7ab-be98d7cff92b admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:12.092242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f31514-3f82-4d44-868d-6e9fc38219dd admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:12.096024 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f31514-3f82-4d44-868d-6e9fc38219dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:12.096705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f31514-3f82-4d44-868d-6e9fc38219dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:12.097460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f31514-3f82-4d44-868d-6e9fc38219dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:12.101748 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f31514-3f82-4d44-868d-6e9fc38219dd admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:12.131586 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1021/2047] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:10:11 2022] PUT /identity/v3/projects/50b9a2f218ee4917b4f6539c345b0eaf/users/52dd55cd6b724e478bbe179f0d1f5f4a/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:10:12.136167 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00bfbf-9530-4b3d-9c4c-cf3f01b307fd None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:12.136167 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00bfbf-9530-4b3d-9c4c-cf3f01b307fd None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:12.136437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00bfbf-9530-4b3d-9c4c-cf3f01b307fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:12.186275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1027/2048] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:10:12 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:10:12.190334 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f31514-3f82-4d44-868d-6e9fc38219dd admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:12.237605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-df00bfbf-9530-4b3d-9c4c-cf3f01b307fd None None] MFA Rules not processed for user `52dd55cd6b724e478bbe179f0d1f5f4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:12.240028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-df00bfbf-9530-4b3d-9c4c-cf3f01b307fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:12.424704 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:12.456262 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f31514-3f82-4d44-868d-6e9fc38219dd admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:12.500429 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a4f1b-43f1-4682-803f-00ae88999266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:12.512062 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1022/2049] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:10:12.512800 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a4f1b-43f1-4682-803f-00ae88999266 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:12.513048 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a4f1b-43f1-4682-803f-00ae88999266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:12.516118 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a4f1b-43f1-4682-803f-00ae88999266 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:12.521585 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04a4f1b-43f1-4682-803f-00ae88999266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:12.540009 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df00bfbf-9530-4b3d-9c4c-cf3f01b307fd None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:12.541913 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-df00bfbf-9530-4b3d-9c4c-cf3f01b307fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:12.563124 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04a4f1b-43f1-4682-803f-00ae88999266 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:12.696726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1028/2050] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:10:12 2022] POST /identity/v3/users => generated 291 bytes in 509 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:10:12.700770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a4f1b-43f1-4682-803f-00ae88999266 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:12.727088 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:12.752183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a4f1b-43f1-4682-803f-00ae88999266 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:12.765850 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e965e94-ed83-4726-8e67-fa44f4949ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:12.768020 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e965e94-ed83-4726-8e67-fa44f4949ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:12.769110 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e965e94-ed83-4726-8e67-fa44f4949ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:12.770620 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e965e94-ed83-4726-8e67-fa44f4949ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:12.772339 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e965e94-ed83-4726-8e67-fa44f4949ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:12.776899 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e965e94-ed83-4726-8e67-fa44f4949ab2 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:12.805437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df00bfbf-9530-4b3d-9c4c-cf3f01b307fd None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:12.813350 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1029/2051] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:10:12.826591 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e965e94-ed83-4726-8e67-fa44f4949ab2 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:12.845973 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb269e-cb42-45c7-ac13-b061d6da699d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a69aeb3fba49b6881f1f15c1758494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:12.860102 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb269e-cb42-45c7-ac13-b061d6da699d service glance] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:12.861076 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb269e-cb42-45c7-ac13-b061d6da699d service glance] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:12.861364 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb269e-cb42-45c7-ac13-b061d6da699d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:12.889173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e965e94-ed83-4726-8e67-fa44f4949ab2 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:12.901727 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb269e-cb42-45c7-ac13-b061d6da699d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:12.911425 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb269e-cb42-45c7-ac13-b061d6da699d service glance] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:12.911613 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b0056-a46b-4fe7-9e9b-c963533f6e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:12.912314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b0056-a46b-4fe7-9e9b-c963533f6e24 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:12.915747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b0056-a46b-4fe7-9e9b-c963533f6e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:12.916077 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b0056-a46b-4fe7-9e9b-c963533f6e24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:12.916980 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b0056-a46b-4fe7-9e9b-c963533f6e24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:12.918612 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b0056-a46b-4fe7-9e9b-c963533f6e24 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:12.947795 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1023/2052] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:10:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:10:12.954008 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1030/2053] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:10:12.954525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb269e-cb42-45c7-ac13-b061d6da699d service glance] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:12.979322 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb269e-cb42-45c7-ac13-b061d6da699d service glance] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:12.993781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb35a6a-68c4-45e3-8062-b735092778f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:12.995088 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb35a6a-68c4-45e3-8062-b735092778f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:12.995382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb35a6a-68c4-45e3-8062-b735092778f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:12.996702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb35a6a-68c4-45e3-8062-b735092778f3 admin admin] PATH_INFO: `/v3/projects/a199c36a2f054a03ae399ace011c3c95/users/0675f7444e544220b53a34dd3086edc5/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:13.062558 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb35a6a-68c4-45e3-8062-b735092778f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a199c36a2f054a03ae399ace011c3c95, user_id=0675f7444e544220b53a34dd3086edc5, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:13.066394 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb35a6a-68c4-45e3-8062-b735092778f3 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:13.116293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1024/2054] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:10:12 2022] PUT /identity/v3/projects/a199c36a2f054a03ae399ace011c3c95/users/0675f7444e544220b53a34dd3086edc5/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:10:13.126195 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d245a28-5d8d-4029-8014-6c866e046b27 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:13.126490 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d245a28-5d8d-4029-8014-6c866e046b27 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:13.126775 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d245a28-5d8d-4029-8014-6c866e046b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:13.201392 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-2d245a28-5d8d-4029-8014-6c866e046b27 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:13.203896 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2d245a28-5d8d-4029-8014-6c866e046b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:13.491315 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1031/2055] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:13.549026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb35a6a-68c4-45e3-8062-b735092778f3 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:13.550916 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1bb35a6a-68c4-45e3-8062-b735092778f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:13.689402 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb35a6a-68c4-45e3-8062-b735092778f3 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:13.699628 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337a22e-37ef-4813-90a6-9dcb719d320d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:13.700307 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337a22e-37ef-4813-90a6-9dcb719d320d service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:13.700684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337a22e-37ef-4813-90a6-9dcb719d320d service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:13.701956 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337a22e-37ef-4813-90a6-9dcb719d320d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:13.730926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337a22e-37ef-4813-90a6-9dcb719d320d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:13.733636 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337a22e-37ef-4813-90a6-9dcb719d320d service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:13.746441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1025/2056] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:10:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:10:15.053761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f0d4f-d701-42c2-964d-e97788557023 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:15.054278 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f0d4f-d701-42c2-964d-e97788557023 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:15.054377 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f0d4f-d701-42c2-964d-e97788557023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:15.143416 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-7c8f0d4f-d701-42c2-964d-e97788557023 None None] MFA Rules not processed for user `f0cf683aa4d14dea974f0febffbbeacc`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:15.145020 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-7c8f0d4f-d701-42c2-964d-e97788557023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:15.334325 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1032/2057] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:15.357824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337a22e-37ef-4813-90a6-9dcb719d320d service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:15.360083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-7337a22e-37ef-4813-90a6-9dcb719d320d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:15.522898 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:15.568500 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337a22e-37ef-4813-90a6-9dcb719d320d service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:15.586330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34daff-2d9b-41f3-b004-38dac3ee9a04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:15.586913 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34daff-2d9b-41f3-b004-38dac3ee9a04 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:15.587199 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34daff-2d9b-41f3-b004-38dac3ee9a04 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:15.587480 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34daff-2d9b-41f3-b004-38dac3ee9a04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:15.613603 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c34daff-2d9b-41f3-b004-38dac3ee9a04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:15.615215 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c34daff-2d9b-41f3-b004-38dac3ee9a04 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:15.635989 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1026/2058] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:10:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:10:15.902056 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4399707-3932-4e6e-bb05-66fead1a6910 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:15.903638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4399707-3932-4e6e-bb05-66fead1a6910 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:15.904139 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4399707-3932-4e6e-bb05-66fead1a6910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:15.959377 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-b4399707-3932-4e6e-bb05-66fead1a6910 None None] MFA Rules not processed for user `0675f7444e544220b53a34dd3086edc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:15.961385 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b4399707-3932-4e6e-bb05-66fead1a6910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:16.157112 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:16.165344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1033/2059] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:16.180685 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34daff-2d9b-41f3-b004-38dac3ee9a04 service nova] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:16.185309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-7c34daff-2d9b-41f3-b004-38dac3ee9a04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:16.337207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34daff-2d9b-41f3-b004-38dac3ee9a04 service nova] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:16.349405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cf2ae-a954-47cf-9e08-a8cb04e20533 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:16.350255 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cf2ae-a954-47cf-9e08-a8cb04e20533 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:16.350690 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cf2ae-a954-47cf-9e08-a8cb04e20533 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:16.351124 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cf2ae-a954-47cf-9e08-a8cb04e20533 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:16.372785 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cf2ae-a954-47cf-9e08-a8cb04e20533 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:16.375339 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cf2ae-a954-47cf-9e08-a8cb04e20533 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:16.408143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1027/2060] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:10:16 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:10:18.269307 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc35d0-1745-4848-86dc-1e7f506bbc08 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:18.269903 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc35d0-1745-4848-86dc-1e7f506bbc08 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:18.269998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc35d0-1745-4848-86dc-1e7f506bbc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:18.341444 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-6afc35d0-1745-4848-86dc-1e7f506bbc08 None None] MFA Rules not processed for user `0675f7444e544220b53a34dd3086edc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:18.348278 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-6afc35d0-1745-4848-86dc-1e7f506bbc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:18.388351 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1034/2061] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:21.250665 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00742d4d-81f7-4436-860a-096913b28775 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:21.251884 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00742d4d-81f7-4436-860a-096913b28775 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:21.254007 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00742d4d-81f7-4436-860a-096913b28775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:21.344756 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-00742d4d-81f7-4436-860a-096913b28775 None None] MFA Rules not processed for user `f0cf683aa4d14dea974f0febffbbeacc`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:21.358039 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-00742d4d-81f7-4436-860a-096913b28775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:21.379785 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1028/2062] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:23.691193 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c04d4-ee54-41a9-a1f1-dee8d7928e90 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:23.692122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c04d4-ee54-41a9-a1f1-dee8d7928e90 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:23.692467 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c04d4-ee54-41a9-a1f1-dee8d7928e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:23.743086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-a26c04d4-ee54-41a9-a1f1-dee8d7928e90 None None] MFA Rules not processed for user `223a5f4ab8914d1c83410fdb2e64390b`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:23.745237 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a26c04d4-ee54-41a9-a1f1-dee8d7928e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:23.875517 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1035/2063] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:23 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:23.889300 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00742d4d-81f7-4436-860a-096913b28775 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:23.892434 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-00742d4d-81f7-4436-860a-096913b28775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:24.023502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00742d4d-81f7-4436-860a-096913b28775 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:24.027634 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2b6b3-6a5c-41a7-9105-02c2bfa41af8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a69aeb3fba49b6881f1f15c1758494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:24.028396 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2b6b3-6a5c-41a7-9105-02c2bfa41af8 service glance] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:24.028715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2b6b3-6a5c-41a7-9105-02c2bfa41af8 service glance] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:24.028994 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2b6b3-6a5c-41a7-9105-02c2bfa41af8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:24.031209 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d7c2b6b3-6a5c-41a7-9105-02c2bfa41af8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:24.104648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2b6b3-6a5c-41a7-9105-02c2bfa41af8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:24.110793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2b6b3-6a5c-41a7-9105-02c2bfa41af8 service glance] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:24.129781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1029/2064] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:10:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:10:24.310423 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a3a39-6096-49f4-9726-d3ea2efe6bc8 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:24.310716 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a3a39-6096-49f4-9726-d3ea2efe6bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:24.311227 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a3a39-6096-49f4-9726-d3ea2efe6bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:24.350928 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-b65a3a39-6096-49f4-9726-d3ea2efe6bc8 None None] MFA Rules not processed for user `7c0356072f92451f98b03a2225e22f86`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:24.352214 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b65a3a39-6096-49f4-9726-d3ea2efe6bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:24.422619 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1036/2065] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 00:10:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:24.476229 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2b6b3-6a5c-41a7-9105-02c2bfa41af8 service glance] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:24.477789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d7c2b6b3-6a5c-41a7-9105-02c2bfa41af8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:24.513318 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:24.554277 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2b6b3-6a5c-41a7-9105-02c2bfa41af8 service glance] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:24.563996 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b21b9-5ab3-4778-ae70-a92c3afc6e71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfd8c7516b7c435d88298672b2596d6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:24.564153 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b21b9-5ab3-4778-ae70-a92c3afc6e71 service swift] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:24.564455 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b21b9-5ab3-4778-ae70-a92c3afc6e71 service swift] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:24.564802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b21b9-5ab3-4778-ae70-a92c3afc6e71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:24.576687 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b21b9-5ab3-4778-ae70-a92c3afc6e71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:24.578809 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b21b9-5ab3-4778-ae70-a92c3afc6e71 service swift] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:24.587911 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1030/2066] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 00:10:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:10:34.415140 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=62 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:34.584727 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=18 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:34.925707 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a3a39-6096-49f4-9726-d3ea2efe6bc8 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:34.928353 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b65a3a39-6096-49f4-9726-d3ea2efe6bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:34.982661 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a3a39-6096-49f4-9726-d3ea2efe6bc8 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:35.004187 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd03baf-ddac-41c0-b810-0dff7e2b34ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:35.005451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd03baf-ddac-41c0-b810-0dff7e2b34ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:35.005884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd03baf-ddac-41c0-b810-0dff7e2b34ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:35.006360 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd03baf-ddac-41c0-b810-0dff7e2b34ad admin admin] PATH_INFO: `/v3/users/18c1e97fc35343e796b9d1122cb1159b` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:35.008425 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd03baf-ddac-41c0-b810-0dff7e2b34ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=18c1e97fc35343e796b9d1122cb1159b)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:35.013038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd03baf-ddac-41c0-b810-0dff7e2b34ad admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:35.174902 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-3bd03baf-ddac-41c0-b810-0dff7e2b34ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18c1e97fc35343e796b9d1122cb1159b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:35.178218 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-3bd03baf-ddac-41c0-b810-0dff7e2b34ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18c1e97fc35343e796b9d1122cb1159b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:35.198946 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-3bd03baf-ddac-41c0-b810-0dff7e2b34ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18c1e97fc35343e796b9d1122cb1159b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:35.235925 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-3bd03baf-ddac-41c0-b810-0dff7e2b34ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18c1e97fc35343e796b9d1122cb1159b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:35.253225 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-3bd03baf-ddac-41c0-b810-0dff7e2b34ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18c1e97fc35343e796b9d1122cb1159b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:35.265886 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1037/2067] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:10:34 2022] DELETE /identity/v3/users/18c1e97fc35343e796b9d1122cb1159b => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:10:35.271755 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b16820-fe43-4e1c-8d71-141a4f6ab96c None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:35.271969 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b16820-fe43-4e1c-8d71-141a4f6ab96c None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:35.272182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b16820-fe43-4e1c-8d71-141a4f6ab96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:35.322063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-a9b16820-fe43-4e1c-8d71-141a4f6ab96c None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:35.328198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a9b16820-fe43-4e1c-8d71-141a4f6ab96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:35.535624 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1031/2068] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:35.576697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd03baf-ddac-41c0-b810-0dff7e2b34ad admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:35.583704 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-3bd03baf-ddac-41c0-b810-0dff7e2b34ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:35.719354 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd03baf-ddac-41c0-b810-0dff7e2b34ad admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:35.724544 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40853dea-4445-49a0-acf3-4f36231be092 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:35.725555 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40853dea-4445-49a0-acf3-4f36231be092 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:35.726198 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40853dea-4445-49a0-acf3-4f36231be092 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:35.726623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40853dea-4445-49a0-acf3-4f36231be092 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:35.744148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40853dea-4445-49a0-acf3-4f36231be092 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:35.748360 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40853dea-4445-49a0-acf3-4f36231be092 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:35.771114 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1038/2069] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:10:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:10:37.382972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b16820-fe43-4e1c-8d71-141a4f6ab96c None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:37.385098 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a9b16820-fe43-4e1c-8d71-141a4f6ab96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:37.417612 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b16820-fe43-4e1c-8d71-141a4f6ab96c None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:37.429795 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c9408-dc4a-41b2-8206-d923d154e395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:37.430734 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c9408-dc4a-41b2-8206-d923d154e395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:37.431262 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c9408-dc4a-41b2-8206-d923d154e395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:37.431695 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c9408-dc4a-41b2-8206-d923d154e395 admin admin] PATH_INFO: `/v3/projects/26754ba666844ff39d7d0e3a67f95d5f` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:37.433389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c9408-dc4a-41b2-8206-d923d154e395 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26754ba666844ff39d7d0e3a67f95d5f)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:37.440643 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c9408-dc4a-41b2-8206-d923d154e395 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:37.520885 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-559c9408-dc4a-41b2-8206-d923d154e395 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26754ba666844ff39d7d0e3a67f95d5f', 'request_id': 'req-559c9408-dc4a-41b2-8206-d923d154e395'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:37.532425 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-559c9408-dc4a-41b2-8206-d923d154e395 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26754ba666844ff39d7d0e3a67f95d5f', 'request_id': 'req-559c9408-dc4a-41b2-8206-d923d154e395'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:37.546983 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-559c9408-dc4a-41b2-8206-d923d154e395 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26754ba666844ff39d7d0e3a67f95d5f', 'request_id': 'req-559c9408-dc4a-41b2-8206-d923d154e395'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:37.564331 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-559c9408-dc4a-41b2-8206-d923d154e395 admin admin] The token cache is being invalidate because project 26754ba666844ff39d7d0e3a67f95d5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:10:37.565337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-559c9408-dc4a-41b2-8206-d923d154e395 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:37.566345 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-559c9408-dc4a-41b2-8206-d923d154e395 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:37.568555 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1032/2070] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:10:37 2022] DELETE /identity/v3/projects/26754ba666844ff39d7d0e3a67f95d5f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:10:37.573798 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a11e9-4b76-459c-ad15-6a19e19fd1d7 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:37.574694 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a11e9-4b76-459c-ad15-6a19e19fd1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:37.575265 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a11e9-4b76-459c-ad15-6a19e19fd1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:37.604797 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-7a6a11e9-4b76-459c-ad15-6a19e19fd1d7 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:37.607082 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-7a6a11e9-4b76-459c-ad15-6a19e19fd1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:37.721996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1039/2071] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:37.729890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d92b8e-d82a-488b-b7b1-4a1ce16c8848 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:37.730727 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d92b8e-d82a-488b-b7b1-4a1ce16c8848 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:37.731102 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d92b8e-d82a-488b-b7b1-4a1ce16c8848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:37.787817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-d5d92b8e-d82a-488b-b7b1-4a1ce16c8848 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:37.792165 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d5d92b8e-d82a-488b-b7b1-4a1ce16c8848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:37.817885 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1033/2072] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:37.823382 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a11e9-4b76-459c-ad15-6a19e19fd1d7 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:37.839227 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a11e9-4b76-459c-ad15-6a19e19fd1d7 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:37.847531 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029ff11-e292-488c-8767-6bf5803f40ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:37.848182 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029ff11-e292-488c-8767-6bf5803f40ac admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:37.848468 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029ff11-e292-488c-8767-6bf5803f40ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:37.849190 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029ff11-e292-488c-8767-6bf5803f40ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:37.850014 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2029ff11-e292-488c-8767-6bf5803f40ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:10:37.850139 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2029ff11-e292-488c-8767-6bf5803f40ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:37.855713 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2029ff11-e292-488c-8767-6bf5803f40ac admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:37.870307 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1040/2073] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:10:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:10:37.874797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d92b8e-d82a-488b-b7b1-4a1ce16c8848 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:37.887351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d92b8e-d82a-488b-b7b1-4a1ce16c8848 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:37.894453 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d07e2d-95af-48b4-8a2e-a7b4f1726fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:37.896307 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d07e2d-95af-48b4-8a2e-a7b4f1726fda admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:37.896307 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d07e2d-95af-48b4-8a2e-a7b4f1726fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:37.896307 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d07e2d-95af-48b4-8a2e-a7b4f1726fda admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:37.896693 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d07e2d-95af-48b4-8a2e-a7b4f1726fda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:37.898604 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d07e2d-95af-48b4-8a2e-a7b4f1726fda admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:37.954757 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1034/2074] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:10:37 2022] POST /identity/v3/projects => generated 397 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:10:37.956757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029ff11-e292-488c-8767-6bf5803f40ac admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:37.962265 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:37.971802 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029ff11-e292-488c-8767-6bf5803f40ac admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:38.077760 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa78af9-2a63-4fb4-a59c-8200f5f3fe2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:38.081169 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa78af9-2a63-4fb4-a59c-8200f5f3fe2d admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:38.081458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa78af9-2a63-4fb4-a59c-8200f5f3fe2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:38.081739 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa78af9-2a63-4fb4-a59c-8200f5f3fe2d admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:38.082401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa78af9-2a63-4fb4-a59c-8200f5f3fe2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:38.084335 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa78af9-2a63-4fb4-a59c-8200f5f3fe2d admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:38.122693 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1041/2075] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:10:37 2022] POST /identity/v3/users => generated 299 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:10:38.125575 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d07e2d-95af-48b4-8a2e-a7b4f1726fda admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:38.160818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d07e2d-95af-48b4-8a2e-a7b4f1726fda admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:38.177867 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214435c-578a-4f55-a519-588901ab01b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:38.179795 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214435c-578a-4f55-a519-588901ab01b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:38.180192 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214435c-578a-4f55-a519-588901ab01b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:38.180683 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214435c-578a-4f55-a519-588901ab01b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:38.190767 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214435c-578a-4f55-a519-588901ab01b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:38.192579 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214435c-578a-4f55-a519-588901ab01b6 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:38.220102 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1035/2076] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:10:38.224464 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa78af9-2a63-4fb4-a59c-8200f5f3fe2d admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:38.236913 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa78af9-2a63-4fb4-a59c-8200f5f3fe2d admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:38.243248 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addfc979-f6a7-4aa7-b84b-4cbca25d9925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:38.246081 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addfc979-f6a7-4aa7-b84b-4cbca25d9925 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:38.246370 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addfc979-f6a7-4aa7-b84b-4cbca25d9925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:38.246651 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addfc979-f6a7-4aa7-b84b-4cbca25d9925 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:38.247321 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addfc979-f6a7-4aa7-b84b-4cbca25d9925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:38.249402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addfc979-f6a7-4aa7-b84b-4cbca25d9925 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:38.272639 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1042/2077] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:10:38.277106 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214435c-578a-4f55-a519-588901ab01b6 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:38.304305 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214435c-578a-4f55-a519-588901ab01b6 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:38.324513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67508a1c-72a7-445b-9019-3f09d06dbbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:38.324513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67508a1c-72a7-445b-9019-3f09d06dbbdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:38.324513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67508a1c-72a7-445b-9019-3f09d06dbbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:38.324513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67508a1c-72a7-445b-9019-3f09d06dbbdb admin admin] PATH_INFO: `/v3/projects/29d7d9e32a3141dc83d09f35a2f9b90d/users/a41034d363e14315a50fb296c15a107e/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:38.351037 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67508a1c-72a7-445b-9019-3f09d06dbbdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=29d7d9e32a3141dc83d09f35a2f9b90d, user_id=a41034d363e14315a50fb296c15a107e, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:38.354209 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67508a1c-72a7-445b-9019-3f09d06dbbdb admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:38.374245 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1036/2078] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:10:38 2022] PUT /identity/v3/projects/29d7d9e32a3141dc83d09f35a2f9b90d/users/a41034d363e14315a50fb296c15a107e/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:10:38.385785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2b50e-3387-4116-bb50-b6517bad2124 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:38.386080 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2b50e-3387-4116-bb50-b6517bad2124 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:38.386351 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2b50e-3387-4116-bb50-b6517bad2124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:38.436522 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-61b2b50e-3387-4116-bb50-b6517bad2124 None None] MFA Rules not processed for user `a41034d363e14315a50fb296c15a107e`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:38.437937 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-61b2b50e-3387-4116-bb50-b6517bad2124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:38.657380 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1043/2079] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:38 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:38.674615 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67508a1c-72a7-445b-9019-3f09d06dbbdb admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:38.680179 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-67508a1c-72a7-445b-9019-3f09d06dbbdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:38.719171 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:38.791535 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67508a1c-72a7-445b-9019-3f09d06dbbdb admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:38.797160 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6064b16-c5eb-49d8-b731-bf9de09d49dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:38.798197 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6064b16-c5eb-49d8-b731-bf9de09d49dd service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:38.798806 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6064b16-c5eb-49d8-b731-bf9de09d49dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:38.799681 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6064b16-c5eb-49d8-b731-bf9de09d49dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:38.818699 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6064b16-c5eb-49d8-b731-bf9de09d49dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:38.820602 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6064b16-c5eb-49d8-b731-bf9de09d49dd service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:38.868159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1037/2080] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:10:38 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:10:39.965995 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad77e1a-1f05-41fc-b0ba-333206579b3f None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:39.966875 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad77e1a-1f05-41fc-b0ba-333206579b3f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:39.967323 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad77e1a-1f05-41fc-b0ba-333206579b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:39.989788 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-1ad77e1a-1f05-41fc-b0ba-333206579b3f None None] MFA Rules not processed for user `fb594a081d194e709023dc7c557afe4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:39.991274 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-1ad77e1a-1f05-41fc-b0ba-333206579b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:40.049446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1044/2081] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:48.844135 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=17 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:50.041980 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:51.520746 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6064b16-c5eb-49d8-b731-bf9de09d49dd service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:51.601633 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6064b16-c5eb-49d8-b731-bf9de09d49dd service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:51.614206 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752e264-a3d8-4cac-b4e6-38e1fdd55487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:51.614830 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752e264-a3d8-4cac-b4e6-38e1fdd55487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:51.615104 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752e264-a3d8-4cac-b4e6-38e1fdd55487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:51.615383 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752e264-a3d8-4cac-b4e6-38e1fdd55487 admin admin] PATH_INFO: `/v3/users/a41034d363e14315a50fb296c15a107e` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:51.616528 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752e264-a3d8-4cac-b4e6-38e1fdd55487 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a41034d363e14315a50fb296c15a107e)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:51.618435 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752e264-a3d8-4cac-b4e6-38e1fdd55487 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:51.670343 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d752e264-a3d8-4cac-b4e6-38e1fdd55487 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a41034d363e14315a50fb296c15a107e'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:51.678336 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d752e264-a3d8-4cac-b4e6-38e1fdd55487 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a41034d363e14315a50fb296c15a107e'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:51.691101 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d752e264-a3d8-4cac-b4e6-38e1fdd55487 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a41034d363e14315a50fb296c15a107e'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:51.691848 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d752e264-a3d8-4cac-b4e6-38e1fdd55487 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a41034d363e14315a50fb296c15a107e'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:51.692456 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d752e264-a3d8-4cac-b4e6-38e1fdd55487 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a41034d363e14315a50fb296c15a107e'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:51.705507 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1038/2082] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:10:51 2022] DELETE /identity/v3/users/a41034d363e14315a50fb296c15a107e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:10:51.709880 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4f9be-db42-4e4a-8a1f-243267f75794 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:51.710347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4f9be-db42-4e4a-8a1f-243267f75794 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:51.710463 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4f9be-db42-4e4a-8a1f-243267f75794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:51.730763 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-a5f4f9be-db42-4e4a-8a1f-243267f75794 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:51.732087 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a5f4f9be-db42-4e4a-8a1f-243267f75794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:51.799390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1045/2083] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:51.808689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752e264-a3d8-4cac-b4e6-38e1fdd55487 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:51.810241 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d752e264-a3d8-4cac-b4e6-38e1fdd55487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:51.871825 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752e264-a3d8-4cac-b4e6-38e1fdd55487 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:51.876557 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51326ff7-ae47-460f-a772-f2c82f0f9096 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:51.877179 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51326ff7-ae47-460f-a772-f2c82f0f9096 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:51.877465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51326ff7-ae47-460f-a772-f2c82f0f9096 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:51.877748 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51326ff7-ae47-460f-a772-f2c82f0f9096 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:51.889074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51326ff7-ae47-460f-a772-f2c82f0f9096 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:51.890529 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51326ff7-ae47-460f-a772-f2c82f0f9096 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:51.905081 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1039/2084] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:10:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:10:52.522479 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4f9be-db42-4e4a-8a1f-243267f75794 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:52.524071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a5f4f9be-db42-4e4a-8a1f-243267f75794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:52.538831 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4f9be-db42-4e4a-8a1f-243267f75794 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:52.544115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696a0e3-7d55-4db2-ad7d-34e6050e8ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:52.544797 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696a0e3-7d55-4db2-ad7d-34e6050e8ed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:52.545086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696a0e3-7d55-4db2-ad7d-34e6050e8ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:52.545369 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696a0e3-7d55-4db2-ad7d-34e6050e8ed0 admin admin] PATH_INFO: `/v3/projects/29d7d9e32a3141dc83d09f35a2f9b90d` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:52.546609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b696a0e3-7d55-4db2-ad7d-34e6050e8ed0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29d7d9e32a3141dc83d09f35a2f9b90d)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:52.548598 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b696a0e3-7d55-4db2-ad7d-34e6050e8ed0 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:52.602350 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b696a0e3-7d55-4db2-ad7d-34e6050e8ed0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29d7d9e32a3141dc83d09f35a2f9b90d', 'request_id': 'req-b696a0e3-7d55-4db2-ad7d-34e6050e8ed0'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:52.612214 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b696a0e3-7d55-4db2-ad7d-34e6050e8ed0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29d7d9e32a3141dc83d09f35a2f9b90d', 'request_id': 'req-b696a0e3-7d55-4db2-ad7d-34e6050e8ed0'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:52.618849 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b696a0e3-7d55-4db2-ad7d-34e6050e8ed0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29d7d9e32a3141dc83d09f35a2f9b90d', 'request_id': 'req-b696a0e3-7d55-4db2-ad7d-34e6050e8ed0'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:52.631808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b696a0e3-7d55-4db2-ad7d-34e6050e8ed0 admin admin] The token cache is being invalidate because project 29d7d9e32a3141dc83d09f35a2f9b90d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:10:52.632101 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b696a0e3-7d55-4db2-ad7d-34e6050e8ed0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:52.633259 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b696a0e3-7d55-4db2-ad7d-34e6050e8ed0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:52.635002 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1046/2085] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:10:52 2022] DELETE /identity/v3/projects/29d7d9e32a3141dc83d09f35a2f9b90d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:10:52.640332 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b03a1-9098-4bf3-8cb8-862f3b3ee234 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:52.641109 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b03a1-9098-4bf3-8cb8-862f3b3ee234 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:52.641109 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b03a1-9098-4bf3-8cb8-862f3b3ee234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:52.663787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-a30b03a1-9098-4bf3-8cb8-862f3b3ee234 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:52.664958 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a30b03a1-9098-4bf3-8cb8-862f3b3ee234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:52.689258 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:52.753846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1040/2086] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:52.755743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d363b81-d09f-4819-bbbc-ac961ebd13f2 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:52.756230 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d363b81-d09f-4819-bbbc-ac961ebd13f2 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:52.756321 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d363b81-d09f-4819-bbbc-ac961ebd13f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:52.779828 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-5d363b81-d09f-4819-bbbc-ac961ebd13f2 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:52.781601 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-5d363b81-d09f-4819-bbbc-ac961ebd13f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:52.791453 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1047/2087] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:52.795916 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b03a1-9098-4bf3-8cb8-862f3b3ee234 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:52.807646 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b03a1-9098-4bf3-8cb8-862f3b3ee234 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:52.813545 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c4158-6c00-4e7d-90bd-b045a3363584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:52.814293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c4158-6c00-4e7d-90bd-b045a3363584 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:52.814670 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c4158-6c00-4e7d-90bd-b045a3363584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:52.814953 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c4158-6c00-4e7d-90bd-b045a3363584 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:52.815735 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c4158-6c00-4e7d-90bd-b045a3363584 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:10:52.816053 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c4158-6c00-4e7d-90bd-b045a3363584 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:52.817843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c4158-6c00-4e7d-90bd-b045a3363584 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:52.831519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1041/2088] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:10:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:10:52.835424 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d363b81-d09f-4819-bbbc-ac961ebd13f2 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:52.845149 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d363b81-d09f-4819-bbbc-ac961ebd13f2 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:52.850055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32f70f-7def-4f2e-8903-d1c9cf31ab4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:52.850751 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32f70f-7def-4f2e-8903-d1c9cf31ab4a admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:52.851036 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32f70f-7def-4f2e-8903-d1c9cf31ab4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:52.851319 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32f70f-7def-4f2e-8903-d1c9cf31ab4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:52.851949 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32f70f-7def-4f2e-8903-d1c9cf31ab4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:52.853854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32f70f-7def-4f2e-8903-d1c9cf31ab4a admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:52.882183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1048/2089] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:10:52 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:10:52.887671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c4158-6c00-4e7d-90bd-b045a3363584 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:52.898568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c4158-6c00-4e7d-90bd-b045a3363584 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:52.956667 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f4fbd3-a7b6-4bb7-beb7-048fc0e65505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:52.957188 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4fbd3-a7b6-4bb7-beb7-048fc0e65505 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:52.957474 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4fbd3-a7b6-4bb7-beb7-048fc0e65505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:52.957752 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4fbd3-a7b6-4bb7-beb7-048fc0e65505 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:52.958382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f4fbd3-a7b6-4bb7-beb7-048fc0e65505 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:52.960269 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f4fbd3-a7b6-4bb7-beb7-048fc0e65505 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:52.995064 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1042/2090] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:10:52 2022] POST /identity/v3/users => generated 286 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:10:52.999092 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32f70f-7def-4f2e-8903-d1c9cf31ab4a admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:53.019916 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32f70f-7def-4f2e-8903-d1c9cf31ab4a admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:53.027981 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6734494-aa0c-4b4f-a7c9-56f6da2ae290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:53.029243 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6734494-aa0c-4b4f-a7c9-56f6da2ae290 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:53.029892 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6734494-aa0c-4b4f-a7c9-56f6da2ae290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:53.031046 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6734494-aa0c-4b4f-a7c9-56f6da2ae290 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:53.032361 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6734494-aa0c-4b4f-a7c9-56f6da2ae290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:53.035581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6734494-aa0c-4b4f-a7c9-56f6da2ae290 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:53.056165 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1049/2091] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:10:53.060822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f4fbd3-a7b6-4bb7-beb7-048fc0e65505 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:53.085161 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f4fbd3-a7b6-4bb7-beb7-048fc0e65505 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:53.093959 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eee927-a7f9-427f-97d5-5038ed10ea8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:53.095170 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eee927-a7f9-427f-97d5-5038ed10ea8b admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:53.096649 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eee927-a7f9-427f-97d5-5038ed10ea8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:53.096649 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eee927-a7f9-427f-97d5-5038ed10ea8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:53.097955 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eee927-a7f9-427f-97d5-5038ed10ea8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:53.101213 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eee927-a7f9-427f-97d5-5038ed10ea8b admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:53.124239 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1043/2092] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:10:53 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:10:53.130276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6734494-aa0c-4b4f-a7c9-56f6da2ae290 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:53.148252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6734494-aa0c-4b4f-a7c9-56f6da2ae290 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:53.156760 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4316b-cecf-4099-b9bb-64de09642c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:53.157806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4316b-cecf-4099-b9bb-64de09642c62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:53.158103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4316b-cecf-4099-b9bb-64de09642c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:53.158385 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4316b-cecf-4099-b9bb-64de09642c62 admin admin] PATH_INFO: `/v3/projects/0401b24ace154000a2b9e80e785f1ce9/users/cd59419dfebe46a28caad71b1454c53a/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:53.181747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4316b-cecf-4099-b9bb-64de09642c62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0401b24ace154000a2b9e80e785f1ce9, user_id=cd59419dfebe46a28caad71b1454c53a, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:53.184721 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4316b-cecf-4099-b9bb-64de09642c62 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:53.208772 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1050/2093] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:10:53 2022] PUT /identity/v3/projects/0401b24ace154000a2b9e80e785f1ce9/users/cd59419dfebe46a28caad71b1454c53a/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:10:53.212390 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1f030-e6c9-4c8e-8d82-694eab455bc3 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:53.212757 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1f030-e6c9-4c8e-8d82-694eab455bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:53.213038 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1f030-e6c9-4c8e-8d82-694eab455bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:53.236314 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-f2f1f030-e6c9-4c8e-8d82-694eab455bc3 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:53.237621 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f2f1f030-e6c9-4c8e-8d82-694eab455bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:53.312332 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1044/2094] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:53.321972 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4316b-cecf-4099-b9bb-64de09642c62 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:53.323686 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-1fd4316b-cecf-4099-b9bb-64de09642c62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:53.347111 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:53.387387 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4316b-cecf-4099-b9bb-64de09642c62 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:53.391735 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0764a9a5-20f4-4837-af18-8a95ec075768 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:53.392392 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764a9a5-20f4-4837-af18-8a95ec075768 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:53.392701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764a9a5-20f4-4837-af18-8a95ec075768 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:53.393040 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764a9a5-20f4-4837-af18-8a95ec075768 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:53.402085 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0764a9a5-20f4-4837-af18-8a95ec075768 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:53.403519 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0764a9a5-20f4-4837-af18-8a95ec075768 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:53.413316 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1051/2095] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:10:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:10:54.746998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4036c29-7ae9-4097-9c3b-4eb21301e9f0 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:54.747936 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4036c29-7ae9-4097-9c3b-4eb21301e9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:54.748283 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4036c29-7ae9-4097-9c3b-4eb21301e9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:54.771948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-c4036c29-7ae9-4097-9c3b-4eb21301e9f0 None None] MFA Rules not processed for user `63286ecd4376471f8ab5f647674eaf43`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:54.773543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-c4036c29-7ae9-4097-9c3b-4eb21301e9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:54.832204 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:54.892079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1045/2096] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:56.914750 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0764a9a5-20f4-4837-af18-8a95ec075768 service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:56.918833 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-0764a9a5-20f4-4837-af18-8a95ec075768 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:56.944733 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0764a9a5-20f4-4837-af18-8a95ec075768 service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:56.958477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedf9dc-1ad7-46a6-8983-f8b2c15dcd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:56.960260 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedf9dc-1ad7-46a6-8983-f8b2c15dcd3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:56.960760 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedf9dc-1ad7-46a6-8983-f8b2c15dcd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:56.960913 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedf9dc-1ad7-46a6-8983-f8b2c15dcd3b admin admin] PATH_INFO: `/v3/users/52dd55cd6b724e478bbe179f0d1f5f4a` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:56.962627 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fedf9dc-1ad7-46a6-8983-f8b2c15dcd3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=52dd55cd6b724e478bbe179f0d1f5f4a)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:56.964919 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fedf9dc-1ad7-46a6-8983-f8b2c15dcd3b admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:57.113394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-6fedf9dc-1ad7-46a6-8983-f8b2c15dcd3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52dd55cd6b724e478bbe179f0d1f5f4a'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:57.116745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed1fc2-6947-4db5-8426-9c15673a5b70 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:57.117151 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed1fc2-6947-4db5-8426-9c15673a5b70 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:57.117335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ed1fc2-6947-4db5-8426-9c15673a5b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:57.120903 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-6fedf9dc-1ad7-46a6-8983-f8b2c15dcd3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52dd55cd6b724e478bbe179f0d1f5f4a'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:57.164626 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-6fedf9dc-1ad7-46a6-8983-f8b2c15dcd3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52dd55cd6b724e478bbe179f0d1f5f4a'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:57.209617 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-a8ed1fc2-6947-4db5-8426-9c15673a5b70 None None] MFA Rules not processed for user `7c0356072f92451f98b03a2225e22f86`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:57.222040 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a8ed1fc2-6947-4db5-8426-9c15673a5b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:57.271318 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-6fedf9dc-1ad7-46a6-8983-f8b2c15dcd3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52dd55cd6b724e478bbe179f0d1f5f4a'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:57.288652 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-6fedf9dc-1ad7-46a6-8983-f8b2c15dcd3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52dd55cd6b724e478bbe179f0d1f5f4a'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:57.302125 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1052/2097] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:10:56 2022] DELETE /identity/v3/users/52dd55cd6b724e478bbe179f0d1f5f4a => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:10:57.307748 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaaddd6-2fe3-4166-97f1-163f0f27fb06 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:57.308056 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaaddd6-2fe3-4166-97f1-163f0f27fb06 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:57.308310 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaaddd6-2fe3-4166-97f1-163f0f27fb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:57.371589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-adaaddd6-2fe3-4166-97f1-163f0f27fb06 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:10:57.379624 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-adaaddd6-2fe3-4166-97f1-163f0f27fb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:57.455778 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1046/2098] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 00:10:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:10:57.462080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed1fc2-6947-4db5-8426-9c15673a5b70 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:57.463831 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a8ed1fc2-6947-4db5-8426-9c15673a5b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:57.666839 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ed1fc2-6947-4db5-8426-9c15673a5b70 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:57.674125 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1053/2099] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:10:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:10:57.741853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaaddd6-2fe3-4166-97f1-163f0f27fb06 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:57.743459 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-adaaddd6-2fe3-4166-97f1-163f0f27fb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:57.817514 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:57.818583 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe917d7-679c-4729-bd2c-e3f5e47e21d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfd8c7516b7c435d88298672b2596d6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:57.819260 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe917d7-679c-4729-bd2c-e3f5e47e21d0 service swift] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:57.819551 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe917d7-679c-4729-bd2c-e3f5e47e21d0 service swift] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:57.821364 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe917d7-679c-4729-bd2c-e3f5e47e21d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:57.849904 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaaddd6-2fe3-4166-97f1-163f0f27fb06 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:57.855939 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6a63f-56ff-4026-8f83-c7c2a86f68ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:57.857244 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6a63f-56ff-4026-8f83-c7c2a86f68ad service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:57.858059 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6a63f-56ff-4026-8f83-c7c2a86f68ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:57.858059 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6a63f-56ff-4026-8f83-c7c2a86f68ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:57.870295 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a6a63f-56ff-4026-8f83-c7c2a86f68ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:57.872608 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a6a63f-56ff-4026-8f83-c7c2a86f68ad service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:57.876373 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:10:57.891765 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1054/2100] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:10:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:10:57.935250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe917d7-679c-4729-bd2c-e3f5e47e21d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:57.938301 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe917d7-679c-4729-bd2c-e3f5e47e21d0 service swift] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:57.957196 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1047/2101] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 00:10:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:10:59.778105 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6a63f-56ff-4026-8f83-c7c2a86f68ad service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:10:59.779709 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-31a6a63f-56ff-4026-8f83-c7c2a86f68ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:10:59.795482 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6a63f-56ff-4026-8f83-c7c2a86f68ad service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:10:59.812952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ee40f-00a1-4053-aaac-c11251da4f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:10:59.813722 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ee40f-00a1-4053-aaac-c11251da4f2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:10:59.814257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ee40f-00a1-4053-aaac-c11251da4f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:10:59.814701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ee40f-00a1-4053-aaac-c11251da4f2f admin admin] PATH_INFO: `/v3/projects/50b9a2f218ee4917b4f6539c345b0eaf` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:10:59.818367 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ee40f-00a1-4053-aaac-c11251da4f2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=50b9a2f218ee4917b4f6539c345b0eaf)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:10:59.820767 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ee40f-00a1-4053-aaac-c11251da4f2f admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:10:59.973458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-e16ee40f-00a1-4053-aaac-c11251da4f2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50b9a2f218ee4917b4f6539c345b0eaf', 'request_id': 'req-e16ee40f-00a1-4053-aaac-c11251da4f2f'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:10:59.990847 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-e16ee40f-00a1-4053-aaac-c11251da4f2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50b9a2f218ee4917b4f6539c345b0eaf', 'request_id': 'req-e16ee40f-00a1-4053-aaac-c11251da4f2f'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:11:00.010464 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-e16ee40f-00a1-4053-aaac-c11251da4f2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50b9a2f218ee4917b4f6539c345b0eaf', 'request_id': 'req-e16ee40f-00a1-4053-aaac-c11251da4f2f'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:11:00.057955 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-e16ee40f-00a1-4053-aaac-c11251da4f2f admin admin] The token cache is being invalidate because project 50b9a2f218ee4917b4f6539c345b0eaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:11:00.058247 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-e16ee40f-00a1-4053-aaac-c11251da4f2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:11:00.064665 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-e16ee40f-00a1-4053-aaac-c11251da4f2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:11:00.066571 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1055/2102] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:10:59 2022] DELETE /identity/v3/projects/50b9a2f218ee4917b4f6539c345b0eaf => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:11:02.767459 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303c52b-a85d-4bf1-a58d-35d6a4b15515 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:11:02.768419 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303c52b-a85d-4bf1-a58d-35d6a4b15515 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:11:02.769149 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303c52b-a85d-4bf1-a58d-35d6a4b15515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:11:02.869448 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-e303c52b-a85d-4bf1-a58d-35d6a4b15515 None None] MFA Rules not processed for user `cd59419dfebe46a28caad71b1454c53a`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:11:02.871290 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e303c52b-a85d-4bf1-a58d-35d6a4b15515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:11:03.201876 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1048/2103] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:11:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 436 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:11:03.206966 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ee40f-00a1-4053-aaac-c11251da4f2f admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:11:03.209240 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-e16ee40f-00a1-4053-aaac-c11251da4f2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:11:03.472692 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ee40f-00a1-4053-aaac-c11251da4f2f admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:11:03.483837 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f77cd-d703-4d8b-9c6c-9e2434e3dead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:11:03.500272 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f77cd-d703-4d8b-9c6c-9e2434e3dead admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:11:03.500689 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f77cd-d703-4d8b-9c6c-9e2434e3dead admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:11:03.500972 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f77cd-d703-4d8b-9c6c-9e2434e3dead admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:11:03.501750 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f77cd-d703-4d8b-9c6c-9e2434e3dead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:11:03.503905 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f77cd-d703-4d8b-9c6c-9e2434e3dead admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:11:03.598455 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1056/2104] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:11:03 2022] POST /identity/v3/projects => generated 373 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:11:03.603101 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303c52b-a85d-4bf1-a58d-35d6a4b15515 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:11:03.629817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303c52b-a85d-4bf1-a58d-35d6a4b15515 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:11:03.772918 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb6da8-8ddc-415b-9a64-facf21444f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:11:03.773692 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb6da8-8ddc-415b-9a64-facf21444f00 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:11:03.773956 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb6da8-8ddc-415b-9a64-facf21444f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:11:03.774312 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb6da8-8ddc-415b-9a64-facf21444f00 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:11:03.775071 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb6da8-8ddc-415b-9a64-facf21444f00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:11:03.777443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb6da8-8ddc-415b-9a64-facf21444f00 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:11:03.814164 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1049/2105] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:11:03 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:11:03.816333 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f77cd-d703-4d8b-9c6c-9e2434e3dead admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:11:03.835925 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f77cd-d703-4d8b-9c6c-9e2434e3dead admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:11:03.843614 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc55fc-5805-4e32-8202-7ada9e59cd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:11:03.844750 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc55fc-5805-4e32-8202-7ada9e59cd53 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:11:03.845248 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc55fc-5805-4e32-8202-7ada9e59cd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:11:03.846196 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc55fc-5805-4e32-8202-7ada9e59cd53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:11:03.847617 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc55fc-5805-4e32-8202-7ada9e59cd53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:11:03.850300 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc55fc-5805-4e32-8202-7ada9e59cd53 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:11:03.878947 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1057/2106] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:11:03.879621 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb6da8-8ddc-415b-9a64-facf21444f00 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:11:03.894605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb6da8-8ddc-415b-9a64-facf21444f00 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:11:03.901561 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3d816-b2b6-4af6-841f-08e89fc9f0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:11:03.902931 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3d816-b2b6-4af6-841f-08e89fc9f0ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:11:03.903556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3d816-b2b6-4af6-841f-08e89fc9f0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:11:03.904174 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3d816-b2b6-4af6-841f-08e89fc9f0ee admin admin] PATH_INFO: `/v3/projects/6fe5f4c42da14e2d8d3fbb270d1be923/users/b65519ee09ab4561b8ae537c0f5d363f/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:11:03.947661 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3d816-b2b6-4af6-841f-08e89fc9f0ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fe5f4c42da14e2d8d3fbb270d1be923, user_id=b65519ee09ab4561b8ae537c0f5d363f, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:11:03.958341 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3d816-b2b6-4af6-841f-08e89fc9f0ee admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:11:03.999791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1050/2107] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:11:03 2022] PUT /identity/v3/projects/6fe5f4c42da14e2d8d3fbb270d1be923/users/b65519ee09ab4561b8ae537c0f5d363f/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:11:04.003968 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc55fc-5805-4e32-8202-7ada9e59cd53 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:11:04.015297 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc55fc-5805-4e32-8202-7ada9e59cd53 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:11:04.054445 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:11:04.083150 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74483e95-6339-4863-9e4a-bee831a4f6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:11:04.083971 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74483e95-6339-4863-9e4a-bee831a4f6eb admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:11:04.084254 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74483e95-6339-4863-9e4a-bee831a4f6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:11:04.084684 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74483e95-6339-4863-9e4a-bee831a4f6eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:11:04.085328 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74483e95-6339-4863-9e4a-bee831a4f6eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:11:04.087338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74483e95-6339-4863-9e4a-bee831a4f6eb admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:11:04.113975 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1058/2108] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:11:04.115727 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3d816-b2b6-4af6-841f-08e89fc9f0ee admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:11:04.129573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3d816-b2b6-4af6-841f-08e89fc9f0ee admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:11:04.140772 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64e924-d4c7-4e43-a455-07fc49b112eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:11:04.141456 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64e924-d4c7-4e43-a455-07fc49b112eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:11:04.141741 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64e924-d4c7-4e43-a455-07fc49b112eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:11:04.142171 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64e924-d4c7-4e43-a455-07fc49b112eb admin admin] PATH_INFO: `/v3/projects/6fe5f4c42da14e2d8d3fbb270d1be923/users/b65519ee09ab4561b8ae537c0f5d363f/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:11:04.144087 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64e924-d4c7-4e43-a455-07fc49b112eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fe5f4c42da14e2d8d3fbb270d1be923, user_id=b65519ee09ab4561b8ae537c0f5d363f, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:11:04.149005 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64e924-d4c7-4e43-a455-07fc49b112eb admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:11:04.177029 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1051/2109] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:11:04 2022] PUT /identity/v3/projects/6fe5f4c42da14e2d8d3fbb270d1be923/users/b65519ee09ab4561b8ae537c0f5d363f/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:11:07.830129 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6483a9-737f-42ca-be11-5a4a90928af0 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:11:07.830739 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6483a9-737f-42ca-be11-5a4a90928af0 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:11:07.830739 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6483a9-737f-42ca-be11-5a4a90928af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:11:07.879192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-6e6483a9-737f-42ca-be11-5a4a90928af0 None None] MFA Rules not processed for user `f95f4fc0d5c5428c96545f543139489b`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:11:07.882608 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-6e6483a9-737f-42ca-be11-5a4a90928af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:11:08.046707 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1059/2110] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:11:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:11:11.302892 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d333775-d2cf-4d04-8f43-b84192febae9 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:11:11.303475 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d333775-d2cf-4d04-8f43-b84192febae9 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:11:11.303745 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d333775-d2cf-4d04-8f43-b84192febae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:11:11.317353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:11:11.360738 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-8d333775-d2cf-4d04-8f43-b84192febae9 None None] MFA Rules not processed for user `b65519ee09ab4561b8ae537c0f5d363f`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:11:11.363014 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-8d333775-d2cf-4d04-8f43-b84192febae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:11:11.528585 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1052/2111] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:11:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:11:11.543429 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6483a9-737f-42ca-be11-5a4a90928af0 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:11:11.545154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-6e6483a9-737f-42ca-be11-5a4a90928af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:11:11.675814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6483a9-737f-42ca-be11-5a4a90928af0 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:11:11.685499 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f487c52-853b-4100-a9be-c4a7b4f5284a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:11:11.686139 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f487c52-853b-4100-a9be-c4a7b4f5284a service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:11:11.686423 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f487c52-853b-4100-a9be-c4a7b4f5284a service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:11:11.687397 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f487c52-853b-4100-a9be-c4a7b4f5284a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:11:11.774870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f487c52-853b-4100-a9be-c4a7b4f5284a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:11:11.776549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f487c52-853b-4100-a9be-c4a7b4f5284a service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:11:11.799190 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1060/2112] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:11:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:11:12.987889 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358344f-2328-4978-9cbb-a7ee2f87a731 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:11:12.988398 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358344f-2328-4978-9cbb-a7ee2f87a731 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:11:12.988480 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c358344f-2328-4978-9cbb-a7ee2f87a731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:11:13.008834 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-c358344f-2328-4978-9cbb-a7ee2f87a731 None None] MFA Rules not processed for user `cd59419dfebe46a28caad71b1454c53a`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:11:13.010107 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-c358344f-2328-4978-9cbb-a7ee2f87a731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:11:13.018515 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1053/2113] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:11:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:11:18.420621 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f487c52-853b-4100-a9be-c4a7b4f5284a service nova] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:11:18.422328 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-0f487c52-853b-4100-a9be-c4a7b4f5284a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:11:18.439364 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:11:18.510070 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f487c52-853b-4100-a9be-c4a7b4f5284a service nova] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:11:18.518947 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc3e75d-6b68-488a-8f63-0719000e5c5b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24108b6355414f64ba5e7bf84011eee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:11:18.519729 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3e75d-6b68-488a-8f63-0719000e5c5b service placement] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:11:18.520019 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3e75d-6b68-488a-8f63-0719000e5c5b service placement] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:11:18.520293 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3e75d-6b68-488a-8f63-0719000e5c5b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:11:18.522276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-7fc3e75d-6b68-488a-8f63-0719000e5c5b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:11:18.536129 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc3e75d-6b68-488a-8f63-0719000e5c5b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:11:18.538049 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc3e75d-6b68-488a-8f63-0719000e5c5b service placement] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:11:18.560446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1061/2114] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:11:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:11:23.003640 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=35 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:11:28.544798 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=26 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:12:07.681428 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05128b74-2d00-4519-910d-83946be90a3a None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:07.681955 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05128b74-2d00-4519-910d-83946be90a3a None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:07.682108 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05128b74-2d00-4519-910d-83946be90a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:07.703094 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-05128b74-2d00-4519-910d-83946be90a3a None None] MFA Rules not processed for user `7c0356072f92451f98b03a2225e22f86`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:12:07.704515 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-05128b74-2d00-4519-910d-83946be90a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:12:07.763870 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1054/2115] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 00:12:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:12:07.810126 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc3e75d-6b68-488a-8f63-0719000e5c5b service placement] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:12:07.812631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-7fc3e75d-6b68-488a-8f63-0719000e5c5b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:12:07.883176 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc3e75d-6b68-488a-8f63-0719000e5c5b service placement] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:12:07.888186 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca55afd-3ffc-4ee3-9289-ba6afc25e677 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfd8c7516b7c435d88298672b2596d6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:12:07.888858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca55afd-3ffc-4ee3-9289-ba6afc25e677 service swift] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:07.889144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca55afd-3ffc-4ee3-9289-ba6afc25e677 service swift] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:07.889422 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca55afd-3ffc-4ee3-9289-ba6afc25e677 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:07.898226 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca55afd-3ffc-4ee3-9289-ba6afc25e677 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:12:07.900153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca55afd-3ffc-4ee3-9289-ba6afc25e677 service swift] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:12:07.909152 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1062/2116] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 00:12:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:12:08.140029 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05128b74-2d00-4519-910d-83946be90a3a None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:12:08.142870 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-05128b74-2d00-4519-910d-83946be90a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:12:08.214526 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05128b74-2d00-4519-910d-83946be90a3a None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:12:08.218981 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9927b36-3b1d-41b9-b9ed-b5b496d4250b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:12:08.220527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9927b36-3b1d-41b9-b9ed-b5b496d4250b service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:08.220940 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9927b36-3b1d-41b9-b9ed-b5b496d4250b service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:08.221237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9927b36-3b1d-41b9-b9ed-b5b496d4250b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:08.223159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b9927b36-3b1d-41b9-b9ed-b5b496d4250b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:12:08.309733 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9927b36-3b1d-41b9-b9ed-b5b496d4250b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:12:08.311307 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9927b36-3b1d-41b9-b9ed-b5b496d4250b service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:12:08.384147 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1055/2117] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:12:08 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:12:17.905261 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=32 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:12:18.320963 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=75 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:12:25.941578 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b1d5d-de11-4978-bbd2-abfcb82aa039 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:25.942070 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b1d5d-de11-4978-bbd2-abfcb82aa039 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:25.942180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b1d5d-de11-4978-bbd2-abfcb82aa039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:25.961534 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-4f1b1d5d-de11-4978-bbd2-abfcb82aa039 None None] MFA Rules not processed for user `7c0356072f92451f98b03a2225e22f86`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:12:25.962719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-4f1b1d5d-de11-4978-bbd2-abfcb82aa039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:12:25.970711 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1063/2118] 10.222.0.13 () {58 vars in 967 bytes} [Sun Oct 23 00:12:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:12:25.984172 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9927b36-3b1d-41b9-b9ed-b5b496d4250b service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:12:25.996563 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9927b36-3b1d-41b9-b9ed-b5b496d4250b service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:12:26.002197 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d89cf54-2d6a-4130-b0ae-5191def7c4df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfd8c7516b7c435d88298672b2596d6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:12:26.002875 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89cf54-2d6a-4130-b0ae-5191def7c4df service swift] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:26.003240 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89cf54-2d6a-4130-b0ae-5191def7c4df service swift] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:26.003577 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89cf54-2d6a-4130-b0ae-5191def7c4df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:26.011957 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d89cf54-2d6a-4130-b0ae-5191def7c4df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:12:26.013625 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d89cf54-2d6a-4130-b0ae-5191def7c4df service swift] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:12:26.022129 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1056/2119] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 00:12:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:12:28.014807 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaae618-37ce-4e99-84c5-fd99fbad14d3 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:28.015299 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaae618-37ce-4e99-84c5-fd99fbad14d3 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:28.015417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaae618-37ce-4e99-84c5-fd99fbad14d3 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:28.017417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1064/2120] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 00:12:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:12:28.026445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4347ee1-80a3-4d19-af7f-247527c40103 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:28.026892 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4347ee1-80a3-4d19-af7f-247527c40103 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:28.027039 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4347ee1-80a3-4d19-af7f-247527c40103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:28.046629 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-b4347ee1-80a3-4d19-af7f-247527c40103 None None] MFA Rules not processed for user `24108b6355414f64ba5e7bf84011eee6`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:12:28.047866 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b4347ee1-80a3-4d19-af7f-247527c40103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:12:28.056838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1057/2121] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 00:12:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:12:28.074474 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaae618-37ce-4e99-84c5-fd99fbad14d3 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:12:28.076186 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-eeaae618-37ce-4e99-84c5-fd99fbad14d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:12:28.090881 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4347ee1-80a3-4d19-af7f-247527c40103 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:12:28.092588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b4347ee1-80a3-4d19-af7f-247527c40103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:12:28.096753 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaae618-37ce-4e99-84c5-fd99fbad14d3 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:12:28.107934 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5d6b5-c115-4c0a-b3e6-4293deca00f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24108b6355414f64ba5e7bf84011eee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:12:28.109036 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5d6b5-c115-4c0a-b3e6-4293deca00f0 service placement] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:28.109333 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5d6b5-c115-4c0a-b3e6-4293deca00f0 service placement] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:28.109612 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5d6b5-c115-4c0a-b3e6-4293deca00f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:28.118796 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de5d6b5-c115-4c0a-b3e6-4293deca00f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:12:28.120417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de5d6b5-c115-4c0a-b3e6-4293deca00f0 service placement] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:12:28.131009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1065/2122] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:12:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:12:28.169252 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4347ee1-80a3-4d19-af7f-247527c40103 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:12:28.174446 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40697ca2-94fd-4a4d-a551-d33c88640cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:12:28.175091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40697ca2-94fd-4a4d-a551-d33c88640cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:28.175488 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40697ca2-94fd-4a4d-a551-d33c88640cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:28.175661 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40697ca2-94fd-4a4d-a551-d33c88640cef admin admin] PATH_INFO: `/v3/users/63286ecd4376471f8ab5f647674eaf43` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:28.176889 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40697ca2-94fd-4a4d-a551-d33c88640cef admin admin] RBAC: Authorizing `identity:delete_user(user_id=63286ecd4376471f8ab5f647674eaf43)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:12:28.178841 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40697ca2-94fd-4a4d-a551-d33c88640cef admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:12:28.225080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-40697ca2-94fd-4a4d-a551-d33c88640cef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63286ecd4376471f8ab5f647674eaf43'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:12:28.231935 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-40697ca2-94fd-4a4d-a551-d33c88640cef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63286ecd4376471f8ab5f647674eaf43'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:12:28.244115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-40697ca2-94fd-4a4d-a551-d33c88640cef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63286ecd4376471f8ab5f647674eaf43'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:12:28.244836 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-40697ca2-94fd-4a4d-a551-d33c88640cef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63286ecd4376471f8ab5f647674eaf43'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:12:28.245473 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-40697ca2-94fd-4a4d-a551-d33c88640cef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63286ecd4376471f8ab5f647674eaf43'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:12:28.258753 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1058/2123] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:12:28 2022] DELETE /identity/v3/users/63286ecd4376471f8ab5f647674eaf43 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:12:28.437920 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5d6b5-c115-4c0a-b3e6-4293deca00f0 service placement] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:12:28.456638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5d6b5-c115-4c0a-b3e6-4293deca00f0 service placement] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:12:28.519900 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dec77a-60c3-4a43-909b-6231764c291a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:12:28.520681 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dec77a-60c3-4a43-909b-6231764c291a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:28.520965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dec77a-60c3-4a43-909b-6231764c291a admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:28.521240 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dec77a-60c3-4a43-909b-6231764c291a admin admin] PATH_INFO: `/v3/projects/bcef7335dafe4b29960c12ce5ab5330b` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:28.522388 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dec77a-60c3-4a43-909b-6231764c291a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcef7335dafe4b29960c12ce5ab5330b)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:12:28.524314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dec77a-60c3-4a43-909b-6231764c291a admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:12:28.578854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-30dec77a-60c3-4a43-909b-6231764c291a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcef7335dafe4b29960c12ce5ab5330b', 'request_id': 'req-30dec77a-60c3-4a43-909b-6231764c291a'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:12:28.587090 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-30dec77a-60c3-4a43-909b-6231764c291a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcef7335dafe4b29960c12ce5ab5330b', 'request_id': 'req-30dec77a-60c3-4a43-909b-6231764c291a'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:12:28.593204 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-30dec77a-60c3-4a43-909b-6231764c291a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcef7335dafe4b29960c12ce5ab5330b', 'request_id': 'req-30dec77a-60c3-4a43-909b-6231764c291a'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:12:28.606333 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-30dec77a-60c3-4a43-909b-6231764c291a admin admin] The token cache is being invalidate because project bcef7335dafe4b29960c12ce5ab5330b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:12:28.606624 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-30dec77a-60c3-4a43-909b-6231764c291a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:12:28.607285 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-30dec77a-60c3-4a43-909b-6231764c291a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:12:28.608932 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1066/2124] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:12:28 2022] DELETE /identity/v3/projects/bcef7335dafe4b29960c12ce5ab5330b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:12:28.744868 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40697ca2-94fd-4a4d-a551-d33c88640cef admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:12:28.746336 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-40697ca2-94fd-4a4d-a551-d33c88640cef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:12:28.815723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40697ca2-94fd-4a4d-a551-d33c88640cef admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:12:28.827714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:12:28.872638 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90389ba9-98a2-4bd7-9016-31b6867375ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:12:28.873243 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90389ba9-98a2-4bd7-9016-31b6867375ee service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:28.873525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90389ba9-98a2-4bd7-9016-31b6867375ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:28.873801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90389ba9-98a2-4bd7-9016-31b6867375ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:28.875507 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-90389ba9-98a2-4bd7-9016-31b6867375ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:12:28.936990 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90389ba9-98a2-4bd7-9016-31b6867375ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:12:28.938634 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90389ba9-98a2-4bd7-9016-31b6867375ee service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:12:28.948116 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1059/2125] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:12:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:12:31.249616 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932b284-89b4-4587-8ff8-f2bce945be72 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:31.251615 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932b284-89b4-4587-8ff8-f2bce945be72 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:31.252327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932b284-89b4-4587-8ff8-f2bce945be72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:31.279136 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-9932b284-89b4-4587-8ff8-f2bce945be72 None None] MFA Rules not processed for user `7c0356072f92451f98b03a2225e22f86`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:12:31.280790 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9932b284-89b4-4587-8ff8-f2bce945be72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:12:31.351318 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1067/2126] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 00:12:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:12:31.366115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90389ba9-98a2-4bd7-9016-31b6867375ee service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:12:31.367700 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-90389ba9-98a2-4bd7-9016-31b6867375ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:12:31.445053 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90389ba9-98a2-4bd7-9016-31b6867375ee service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:12:31.451042 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ed0ee-9556-4bea-af2c-0dbcfaf5bfcb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfd8c7516b7c435d88298672b2596d6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:12:31.451658 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ed0ee-9556-4bea-af2c-0dbcfaf5bfcb service swift] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:31.451938 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ed0ee-9556-4bea-af2c-0dbcfaf5bfcb service swift] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:31.452213 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ed0ee-9556-4bea-af2c-0dbcfaf5bfcb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:31.462910 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ed0ee-9556-4bea-af2c-0dbcfaf5bfcb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:12:31.464760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ed0ee-9556-4bea-af2c-0dbcfaf5bfcb service swift] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:12:31.474026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1060/2127] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 00:12:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:12:31.965610 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e3d48-7f0e-4123-84aa-773c6cad1299 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:31.965987 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e3d48-7f0e-4123-84aa-773c6cad1299 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:31.966343 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e3d48-7f0e-4123-84aa-773c6cad1299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:31.990957 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-bf2e3d48-7f0e-4123-84aa-773c6cad1299 None None] MFA Rules not processed for user `0675f7444e544220b53a34dd3086edc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:12:31.992333 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-bf2e3d48-7f0e-4123-84aa-773c6cad1299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:12:32.002745 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:12:32.055891 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1068/2128] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:12:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:12:41.469833 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=75 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:12:42.048624 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=18 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:12:46.528224 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54da3f7-f57c-4aa8-a0ae-314cd460ab08 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:46.528916 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54da3f7-f57c-4aa8-a0ae-314cd460ab08 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:46.528916 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54da3f7-f57c-4aa8-a0ae-314cd460ab08 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:46.530272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1061/2129] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 00:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:12:46.536574 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689776e-f01c-4c30-aa30-57e267b402b9 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:46.537083 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689776e-f01c-4c30-aa30-57e267b402b9 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:46.537190 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689776e-f01c-4c30-aa30-57e267b402b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:46.559400 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-b689776e-f01c-4c30-aa30-57e267b402b9 None None] MFA Rules not processed for user `c4a3579d56294b9d8ad36d4d174c8526`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:12:46.560798 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b689776e-f01c-4c30-aa30-57e267b402b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:12:46.638894 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1069/2130] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 00:12:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:12:46.651799 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54da3f7-f57c-4aa8-a0ae-314cd460ab08 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:12:46.653919 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d54da3f7-f57c-4aa8-a0ae-314cd460ab08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:12:46.727658 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54da3f7-f57c-4aa8-a0ae-314cd460ab08 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:12:46.731314 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd32fc-2c50-4998-af22-69ecc019af4d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a69aeb3fba49b6881f1f15c1758494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:12:46.731993 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd32fc-2c50-4998-af22-69ecc019af4d service glance] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:46.732378 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd32fc-2c50-4998-af22-69ecc019af4d service glance] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:46.732747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd32fc-2c50-4998-af22-69ecc019af4d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:46.743479 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd32fc-2c50-4998-af22-69ecc019af4d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:12:46.745014 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd32fc-2c50-4998-af22-69ecc019af4d service glance] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:12:46.756065 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1062/2131] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:12:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:12:46.825719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e87d58-1b16-4ee7-ab70-11ab50525643 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:46.826004 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e87d58-1b16-4ee7-ab70-11ab50525643 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:46.826298 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e87d58-1b16-4ee7-ab70-11ab50525643 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:46.827641 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1070/2132] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 00:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:12:46.832330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e567d4-fb5e-43e7-ae87-a210fa85f344 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:46.832666 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e567d4-fb5e-43e7-ae87-a210fa85f344 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:46.832952 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e567d4-fb5e-43e7-ae87-a210fa85f344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:46.853315 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-53e567d4-fb5e-43e7-ae87-a210fa85f344 None None] MFA Rules not processed for user `a09d45d080bc461c88f38ff281b75b06`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:12:46.854508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-53e567d4-fb5e-43e7-ae87-a210fa85f344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:12:46.861920 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1063/2133] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 00:12:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:12:46.873885 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e87d58-1b16-4ee7-ab70-11ab50525643 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:12:46.875576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a2e87d58-1b16-4ee7-ab70-11ab50525643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:12:46.888978 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e87d58-1b16-4ee7-ab70-11ab50525643 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:12:46.892687 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83fb6df-47f7-4f4c-8c6d-745d6f64ca75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:12:46.893289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fb6df-47f7-4f4c-8c6d-745d6f64ca75 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:46.893569 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fb6df-47f7-4f4c-8c6d-745d6f64ca75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:46.893855 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83fb6df-47f7-4f4c-8c6d-745d6f64ca75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:46.900993 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83fb6df-47f7-4f4c-8c6d-745d6f64ca75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:12:46.902396 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83fb6df-47f7-4f4c-8c6d-745d6f64ca75 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:12:46.910532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1071/2134] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:12:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:12:46.995572 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b969b8-9c9e-4528-8bc0-8c4231b6f741 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:46.996013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b969b8-9c9e-4528-8bc0-8c4231b6f741 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:46.996301 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b969b8-9c9e-4528-8bc0-8c4231b6f741 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:46.997698 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1064/2135] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 00:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:12:47.003682 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96336144-0419-428b-888b-c5377acc8b9f None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:47.003968 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96336144-0419-428b-888b-c5377acc8b9f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:47.004758 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96336144-0419-428b-888b-c5377acc8b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:47.024897 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-96336144-0419-428b-888b-c5377acc8b9f None None] MFA Rules not processed for user `c4a3579d56294b9d8ad36d4d174c8526`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:12:47.026104 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-96336144-0419-428b-888b-c5377acc8b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:12:47.035996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1072/2136] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 00:12:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:12:47.046331 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b969b8-9c9e-4528-8bc0-8c4231b6f741 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:12:47.059973 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b969b8-9c9e-4528-8bc0-8c4231b6f741 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:12:47.064631 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04143c4b-8c9f-4d28-8a4d-12974fd32cd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:12:47.065405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04143c4b-8c9f-4d28-8a4d-12974fd32cd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:47.065685 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04143c4b-8c9f-4d28-8a4d-12974fd32cd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:47.065952 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04143c4b-8c9f-4d28-8a4d-12974fd32cd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:47.079228 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04143c4b-8c9f-4d28-8a4d-12974fd32cd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:12:47.081637 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04143c4b-8c9f-4d28-8a4d-12974fd32cd1 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:12:47.104907 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1065/2137] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:12:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:12:49.933697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a73c9-078e-49ef-bcf3-a634e0de4841 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:49.934166 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a73c9-078e-49ef-bcf3-a634e0de4841 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:49.934265 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a73c9-078e-49ef-bcf3-a634e0de4841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:49.953803 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-ea0a73c9-078e-49ef-bcf3-a634e0de4841 None None] MFA Rules not processed for user `7c0356072f92451f98b03a2225e22f86`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:12:49.955660 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ea0a73c9-078e-49ef-bcf3-a634e0de4841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:12:49.962821 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1073/2138] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 00:12:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:12:49.974637 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04143c4b-8c9f-4d28-8a4d-12974fd32cd1 service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:12:49.985976 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04143c4b-8c9f-4d28-8a4d-12974fd32cd1 service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:12:49.991325 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb1351-d569-4bb8-aa5a-87c633643b65 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfd8c7516b7c435d88298672b2596d6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:12:49.991915 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb1351-d569-4bb8-aa5a-87c633643b65 service swift] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:49.992320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb1351-d569-4bb8-aa5a-87c633643b65 service swift] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:49.992633 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb1351-d569-4bb8-aa5a-87c633643b65 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:50.000367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb1351-d569-4bb8-aa5a-87c633643b65 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:12:50.002854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb1351-d569-4bb8-aa5a-87c633643b65 service swift] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:12:50.010945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1066/2139] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 00:12:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:12:52.578504 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a73c9-078e-49ef-bcf3-a634e0de4841 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:12:52.580026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ea0a73c9-078e-49ef-bcf3-a634e0de4841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:12:52.591175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a73c9-078e-49ef-bcf3-a634e0de4841 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:12:52.594700 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710068d-9cbb-4e89-ab30-0939805f3344 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:12:52.595293 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710068d-9cbb-4e89-ab30-0939805f3344 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:52.595617 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710068d-9cbb-4e89-ab30-0939805f3344 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:52.596038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710068d-9cbb-4e89-ab30-0939805f3344 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:52.597708 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-7710068d-9cbb-4e89-ab30-0939805f3344 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:12:52.611494 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710068d-9cbb-4e89-ab30-0939805f3344 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:12:52.613180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710068d-9cbb-4e89-ab30-0939805f3344 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:12:52.623831 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1074/2140] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:12:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:12:53.820544 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb1351-d569-4bb8-aa5a-87c633643b65 service swift] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:12:53.829933 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb1351-d569-4bb8-aa5a-87c633643b65 service swift] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:12:53.833666 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4137b-74a6-4eb5-8ff5-53263f888e45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:12:53.834272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4137b-74a6-4eb5-8ff5-53263f888e45 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:12:53.834555 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4137b-74a6-4eb5-8ff5-53263f888e45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:12:53.834849 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4137b-74a6-4eb5-8ff5-53263f888e45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:12:53.836664 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-92d4137b-74a6-4eb5-8ff5-53263f888e45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:12:53.852011 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d4137b-74a6-4eb5-8ff5-53263f888e45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:12:53.853517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d4137b-74a6-4eb5-8ff5-53263f888e45 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:12:53.865555 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1067/2141] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:12:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:13:02.137977 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710068d-9cbb-4e89-ab30-0939805f3344 service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:13:02.139359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-7710068d-9cbb-4e89-ab30-0939805f3344 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:13:02.153156 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710068d-9cbb-4e89-ab30-0939805f3344 service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:13:02.159412 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d64e93-95bc-4c0d-bd7f-8b93f87d08b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:13:02.159848 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d64e93-95bc-4c0d-bd7f-8b93f87d08b8 service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:13:02.160132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d64e93-95bc-4c0d-bd7f-8b93f87d08b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:13:02.160412 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d64e93-95bc-4c0d-bd7f-8b93f87d08b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:13:02.162178 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-10d64e93-95bc-4c0d-bd7f-8b93f87d08b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:13:02.223439 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d64e93-95bc-4c0d-bd7f-8b93f87d08b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:13:02.225054 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d64e93-95bc-4c0d-bd7f-8b93f87d08b8 service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:13:02.234583 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1075/2142] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:13:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:13:03.859908 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=47 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:13:12.230020 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=77 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:13:17.107895 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4137b-74a6-4eb5-8ff5-53263f888e45 service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:13:17.117304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d4137b-74a6-4eb5-8ff5-53263f888e45 service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:13:17.121086 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf279e7-0ab1-4d30-ae60-75c890d1ef41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:13:17.121722 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf279e7-0ab1-4d30-ae60-75c890d1ef41 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:13:17.122010 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf279e7-0ab1-4d30-ae60-75c890d1ef41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:13:17.122290 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf279e7-0ab1-4d30-ae60-75c890d1ef41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:13:17.124021 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ebf279e7-0ab1-4d30-ae60-75c890d1ef41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:13:17.134352 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf279e7-0ab1-4d30-ae60-75c890d1ef41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:13:17.135818 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf279e7-0ab1-4d30-ae60-75c890d1ef41 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:13:17.144211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1068/2143] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:13:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:13:27.140345 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=6 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:13:36.837833 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c88a5-2b50-4558-a816-4df28fee7409 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:13:36.838352 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c88a5-2b50-4558-a816-4df28fee7409 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:13:36.838453 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c88a5-2b50-4558-a816-4df28fee7409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:13:36.858497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-001c88a5-2b50-4558-a816-4df28fee7409 None None] MFA Rules not processed for user `7c0356072f92451f98b03a2225e22f86`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:13:36.859692 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-001c88a5-2b50-4558-a816-4df28fee7409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:13:36.867674 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1076/2144] 10.222.0.13 () {58 vars in 967 bytes} [Sun Oct 23 00:13:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:13:36.890235 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf279e7-0ab1-4d30-ae60-75c890d1ef41 service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:13:36.903194 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf279e7-0ab1-4d30-ae60-75c890d1ef41 service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:13:36.908088 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5841d-364b-472d-a91f-f2856cc0da9b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfd8c7516b7c435d88298672b2596d6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:13:36.908702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5841d-364b-472d-a91f-f2856cc0da9b service swift] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:13:36.908982 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5841d-364b-472d-a91f-f2856cc0da9b service swift] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:13:36.909257 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5841d-364b-472d-a91f-f2856cc0da9b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:13:36.918046 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5841d-364b-472d-a91f-f2856cc0da9b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:13:36.919616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5841d-364b-472d-a91f-f2856cc0da9b service swift] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:13:36.927908 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1069/2145] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 00:13:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:13:40.776286 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c88a5-2b50-4558-a816-4df28fee7409 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:13:40.778592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-001c88a5-2b50-4558-a816-4df28fee7409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:13:40.801186 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c88a5-2b50-4558-a816-4df28fee7409 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:13:40.807949 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb9a7d-861f-46b8-942b-6f0893adda02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:13:40.808889 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb9a7d-861f-46b8-942b-6f0893adda02 service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:13:40.810008 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb9a7d-861f-46b8-942b-6f0893adda02 service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:13:40.810665 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb9a7d-861f-46b8-942b-6f0893adda02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:13:40.813096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-0acb9a7d-861f-46b8-942b-6f0893adda02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:13:40.827615 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acb9a7d-861f-46b8-942b-6f0893adda02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:13:40.829398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acb9a7d-861f-46b8-942b-6f0893adda02 service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:13:40.841747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1077/2146] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:13:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:13:46.924735 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=6 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:13:50.836683 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=9 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:14:09.844367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5841d-364b-472d-a91f-f2856cc0da9b service swift] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:14:09.845997 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5ad5841d-364b-472d-a91f-f2856cc0da9b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:14:09.911134 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5841d-364b-472d-a91f-f2856cc0da9b service swift] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:14:09.915697 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812a92c-c844-4f96-9469-88696274bf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:14:09.917113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812a92c-c844-4f96-9469-88696274bf09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:14:09.917394 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812a92c-c844-4f96-9469-88696274bf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:14:09.917667 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4812a92c-c844-4f96-9469-88696274bf09 admin admin] PATH_INFO: `/v3/users/0675f7444e544220b53a34dd3086edc5` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:14:09.918791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4812a92c-c844-4f96-9469-88696274bf09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0675f7444e544220b53a34dd3086edc5)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:14:09.920715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4812a92c-c844-4f96-9469-88696274bf09 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:14:09.967022 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4812a92c-c844-4f96-9469-88696274bf09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0675f7444e544220b53a34dd3086edc5'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:14:09.974026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4812a92c-c844-4f96-9469-88696274bf09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0675f7444e544220b53a34dd3086edc5'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:14:09.986719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4812a92c-c844-4f96-9469-88696274bf09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0675f7444e544220b53a34dd3086edc5'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:14:09.987405 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4812a92c-c844-4f96-9469-88696274bf09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0675f7444e544220b53a34dd3086edc5'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:14:09.988023 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4812a92c-c844-4f96-9469-88696274bf09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0675f7444e544220b53a34dd3086edc5'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:14:10.000035 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1070/2147] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:14:09 2022] DELETE /identity/v3/users/0675f7444e544220b53a34dd3086edc5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:14:10.173888 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb9a7d-861f-46b8-942b-6f0893adda02 service nova] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:14:10.185761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb9a7d-861f-46b8-942b-6f0893adda02 service nova] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:14:10.245386 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4c20d-8e4b-4f1d-b562-cbc1cef4f434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:14:10.246015 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4c20d-8e4b-4f1d-b562-cbc1cef4f434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:14:10.246296 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4c20d-8e4b-4f1d-b562-cbc1cef4f434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:14:10.246571 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4c20d-8e4b-4f1d-b562-cbc1cef4f434 admin admin] PATH_INFO: `/v3/projects/a199c36a2f054a03ae399ace011c3c95` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:14:10.247691 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4c20d-8e4b-4f1d-b562-cbc1cef4f434 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a199c36a2f054a03ae399ace011c3c95)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:14:10.249612 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a4c20d-8e4b-4f1d-b562-cbc1cef4f434 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:14:10.298178 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-60a4c20d-8e4b-4f1d-b562-cbc1cef4f434 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a199c36a2f054a03ae399ace011c3c95', 'request_id': 'req-60a4c20d-8e4b-4f1d-b562-cbc1cef4f434'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:14:10.306165 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-60a4c20d-8e4b-4f1d-b562-cbc1cef4f434 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a199c36a2f054a03ae399ace011c3c95', 'request_id': 'req-60a4c20d-8e4b-4f1d-b562-cbc1cef4f434'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:14:10.312899 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-60a4c20d-8e4b-4f1d-b562-cbc1cef4f434 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a199c36a2f054a03ae399ace011c3c95', 'request_id': 'req-60a4c20d-8e4b-4f1d-b562-cbc1cef4f434'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:14:10.324915 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-60a4c20d-8e4b-4f1d-b562-cbc1cef4f434 admin admin] The token cache is being invalidate because project a199c36a2f054a03ae399ace011c3c95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:14:10.325215 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-60a4c20d-8e4b-4f1d-b562-cbc1cef4f434 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:14:10.325861 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-60a4c20d-8e4b-4f1d-b562-cbc1cef4f434 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:14:10.327360 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1078/2148] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:14:10 2022] DELETE /identity/v3/projects/a199c36a2f054a03ae399ace011c3c95 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:14:19.996633 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:14:20.321044 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:14:23.143589 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812a92c-c844-4f96-9469-88696274bf09 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:14:23.145072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-4812a92c-c844-4f96-9469-88696274bf09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:14:23.220409 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4812a92c-c844-4f96-9469-88696274bf09 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:14:23.290514 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeda415-f511-48a1-b8a6-f44372cea9f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:14:23.291825 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeda415-f511-48a1-b8a6-f44372cea9f0 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:14:23.292034 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeda415-f511-48a1-b8a6-f44372cea9f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:14:23.292312 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeda415-f511-48a1-b8a6-f44372cea9f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:14:23.294175 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5eeda415-f511-48a1-b8a6-f44372cea9f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:14:23.352679 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeda415-f511-48a1-b8a6-f44372cea9f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:14:23.355340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeda415-f511-48a1-b8a6-f44372cea9f0 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:14:23.364986 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1071/2149] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:14:23 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:14:33.360497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=84 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:14:46.823438 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb350c-da12-476e-baca-5c508e21abcf None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:14:46.824117 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb350c-da12-476e-baca-5c508e21abcf None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:14:46.824117 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb350c-da12-476e-baca-5c508e21abcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:14:46.843295 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-b0cb350c-da12-476e-baca-5c508e21abcf None None] MFA Rules not processed for user `f95f4fc0d5c5428c96545f543139489b`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:14:46.844635 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b0cb350c-da12-476e-baca-5c508e21abcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:14:46.851516 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1079/2150] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:14:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:14:53.022649 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeda415-f511-48a1-b8a6-f44372cea9f0 service nova] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:14:53.024207 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5eeda415-f511-48a1-b8a6-f44372cea9f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:14:53.041493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeda415-f511-48a1-b8a6-f44372cea9f0 service nova] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:14:53.050223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34c5b7-f37e-411e-8434-797ef4705d39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:14:53.051008 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34c5b7-f37e-411e-8434-797ef4705d39 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:14:53.051294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34c5b7-f37e-411e-8434-797ef4705d39 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:14:53.051573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34c5b7-f37e-411e-8434-797ef4705d39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:14:53.053329 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-fa34c5b7-f37e-411e-8434-797ef4705d39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:14:53.065698 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34c5b7-f37e-411e-8434-797ef4705d39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:14:53.067263 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34c5b7-f37e-411e-8434-797ef4705d39 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:14:53.078477 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1072/2151] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:14:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:14:56.837880 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:14:59.415424 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79121ba-9e96-42f6-8354-33b7c9078474 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:14:59.415945 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79121ba-9e96-42f6-8354-33b7c9078474 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:14:59.416050 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79121ba-9e96-42f6-8354-33b7c9078474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:14:59.435248 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-f79121ba-9e96-42f6-8354-33b7c9078474 None None] MFA Rules not processed for user `8b4ef165353f4f1189400c005519c2ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:14:59.436443 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f79121ba-9e96-42f6-8354-33b7c9078474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:14:59.492408 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1080/2152] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:14:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:14:59.799765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34c5b7-f37e-411e-8434-797ef4705d39 service nova] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:14:59.801189 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-fa34c5b7-f37e-411e-8434-797ef4705d39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:14:59.858908 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34c5b7-f37e-411e-8434-797ef4705d39 service nova] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:14:59.862537 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a26fd-35b9-4087-ba3d-00dd502615ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:14:59.863066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a26fd-35b9-4087-ba3d-00dd502615ed service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:14:59.863347 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a26fd-35b9-4087-ba3d-00dd502615ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:14:59.863618 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a26fd-35b9-4087-ba3d-00dd502615ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:14:59.865320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-853a26fd-35b9-4087-ba3d-00dd502615ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:14:59.875370 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a26fd-35b9-4087-ba3d-00dd502615ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:14:59.876842 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a26fd-35b9-4087-ba3d-00dd502615ed service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:14:59.884601 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1073/2153] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:14:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:15:00.199375 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79121ba-9e96-42f6-8354-33b7c9078474 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:15:00.200903 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f79121ba-9e96-42f6-8354-33b7c9078474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:15:00.273243 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79121ba-9e96-42f6-8354-33b7c9078474 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:15:00.279839 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dff763-64c2-40d5-99ac-c5498abf1658 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24108b6355414f64ba5e7bf84011eee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:15:00.280451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dff763-64c2-40d5-99ac-c5498abf1658 service placement] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:15:00.280760 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dff763-64c2-40d5-99ac-c5498abf1658 service placement] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:15:00.281032 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dff763-64c2-40d5-99ac-c5498abf1658 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:15:00.282730 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-13dff763-64c2-40d5-99ac-c5498abf1658 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:15:00.295156 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dff763-64c2-40d5-99ac-c5498abf1658 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:15:00.296778 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dff763-64c2-40d5-99ac-c5498abf1658 service placement] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:15:00.306682 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1081/2154] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:15:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:15:09.881137 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=32 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:15:10.302543 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=57 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:15:10.522959 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a26fd-35b9-4087-ba3d-00dd502615ed service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:15:10.524473 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-853a26fd-35b9-4087-ba3d-00dd502615ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:15:10.541154 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a26fd-35b9-4087-ba3d-00dd502615ed service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:15:10.547329 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479ffdb-a118-4ba5-9dd2-18f1ea0909a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:15:10.547992 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479ffdb-a118-4ba5-9dd2-18f1ea0909a1 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:15:10.548302 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479ffdb-a118-4ba5-9dd2-18f1ea0909a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:15:10.548683 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479ffdb-a118-4ba5-9dd2-18f1ea0909a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:15:10.550411 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-2479ffdb-a118-4ba5-9dd2-18f1ea0909a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:15:10.563871 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479ffdb-a118-4ba5-9dd2-18f1ea0909a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:15:10.565564 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479ffdb-a118-4ba5-9dd2-18f1ea0909a1 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:15:10.577630 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1074/2155] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:15:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:15:20.571669 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=6 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:15:33.322615 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dff763-64c2-40d5-99ac-c5498abf1658 service placement] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:15:33.335674 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dff763-64c2-40d5-99ac-c5498abf1658 service placement] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:15:33.341347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d3920-d894-4ad8-b6fd-1377cc74fc5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:15:33.342036 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d3920-d894-4ad8-b6fd-1377cc74fc5f service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:15:33.342382 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d3920-d894-4ad8-b6fd-1377cc74fc5f service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:15:33.343661 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d3920-d894-4ad8-b6fd-1377cc74fc5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:15:33.345278 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-408d3920-d894-4ad8-b6fd-1377cc74fc5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:15:33.403920 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d3920-d894-4ad8-b6fd-1377cc74fc5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:15:33.406330 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d3920-d894-4ad8-b6fd-1377cc74fc5f service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:15:33.415140 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1082/2156] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:15:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:15:41.218589 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479ffdb-a118-4ba5-9dd2-18f1ea0909a1 service nova] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:15:41.229665 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479ffdb-a118-4ba5-9dd2-18f1ea0909a1 service nova] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:15:41.235667 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d0ccb6-70f8-4737-ac71-6666c5732474 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:15:41.236256 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0ccb6-70f8-4737-ac71-6666c5732474 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:15:41.236574 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0ccb6-70f8-4737-ac71-6666c5732474 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:15:41.236853 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0ccb6-70f8-4737-ac71-6666c5732474 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:15:41.238567 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-36d0ccb6-70f8-4737-ac71-6666c5732474 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:15:41.295653 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d0ccb6-70f8-4737-ac71-6666c5732474 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:15:41.297282 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d0ccb6-70f8-4737-ac71-6666c5732474 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:15:41.306030 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1075/2157] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:15:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:15:43.411081 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=28 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:15:50.476639 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb1748-b97b-4bca-98ab-f6218c279f4c None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:15:50.477205 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb1748-b97b-4bca-98ab-f6218c279f4c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:15:50.477205 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb1748-b97b-4bca-98ab-f6218c279f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:15:50.496397 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-18cb1748-b97b-4bca-98ab-f6218c279f4c None None] MFA Rules not processed for user `7c0356072f92451f98b03a2225e22f86`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:15:50.497624 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-18cb1748-b97b-4bca-98ab-f6218c279f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:15:50.552511 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1083/2158] 10.222.0.13 () {58 vars in 967 bytes} [Sun Oct 23 00:15:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:15:50.597700 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d0ccb6-70f8-4737-ac71-6666c5732474 service nova] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:15:50.599124 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-36d0ccb6-70f8-4737-ac71-6666c5732474 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:15:50.668208 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d0ccb6-70f8-4737-ac71-6666c5732474 service nova] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:15:50.674034 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac751f3-6184-4dca-90be-c704bdc219a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfd8c7516b7c435d88298672b2596d6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:15:50.674628 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac751f3-6184-4dca-90be-c704bdc219a3 service swift] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:15:50.674908 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac751f3-6184-4dca-90be-c704bdc219a3 service swift] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:15:50.675181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac751f3-6184-4dca-90be-c704bdc219a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:15:50.683097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac751f3-6184-4dca-90be-c704bdc219a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:15:50.684672 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac751f3-6184-4dca-90be-c704bdc219a3 service swift] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:15:50.693291 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1076/2159] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 00:15:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:15:51.016553 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb1748-b97b-4bca-98ab-f6218c279f4c None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:15:51.017960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-18cb1748-b97b-4bca-98ab-f6218c279f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:15:51.029343 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb1748-b97b-4bca-98ab-f6218c279f4c None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:15:51.033784 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671db6bb-f7a0-470d-845a-d16b1b8e6d61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:15:51.034385 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671db6bb-f7a0-470d-845a-d16b1b8e6d61 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:15:51.034669 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671db6bb-f7a0-470d-845a-d16b1b8e6d61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:15:51.034942 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671db6bb-f7a0-470d-845a-d16b1b8e6d61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:15:51.036639 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-671db6bb-f7a0-470d-845a-d16b1b8e6d61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:15:51.098957 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671db6bb-f7a0-470d-845a-d16b1b8e6d61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:15:51.100373 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671db6bb-f7a0-470d-845a-d16b1b8e6d61 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:15:51.111111 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1084/2160] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:15:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:15:57.894174 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac751f3-6184-4dca-90be-c704bdc219a3 service swift] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:15:57.939070 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac751f3-6184-4dca-90be-c704bdc219a3 service swift] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:15:57.944499 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff2dcd-999d-400b-9613-a4ff7b23e59f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24108b6355414f64ba5e7bf84011eee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:15:57.945138 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff2dcd-999d-400b-9613-a4ff7b23e59f service placement] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:15:57.945425 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff2dcd-999d-400b-9613-a4ff7b23e59f service placement] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:15:57.945704 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff2dcd-999d-400b-9613-a4ff7b23e59f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:15:57.950047 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-61ff2dcd-999d-400b-9613-a4ff7b23e59f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:15:57.952472 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671db6bb-f7a0-470d-845a-d16b1b8e6d61 service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:15:57.954509 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-671db6bb-f7a0-470d-845a-d16b1b8e6d61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:15:57.984549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff2dcd-999d-400b-9613-a4ff7b23e59f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:15:57.986185 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff2dcd-999d-400b-9613-a4ff7b23e59f service placement] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:15:57.996349 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1077/2161] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:15:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:15:58.046820 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671db6bb-f7a0-470d-845a-d16b1b8e6d61 service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:15:58.051380 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75000915-57bf-498d-a557-c8e5eca3117d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:15:58.052004 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75000915-57bf-498d-a557-c8e5eca3117d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:15:58.052287 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75000915-57bf-498d-a557-c8e5eca3117d admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:15:58.052589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75000915-57bf-498d-a557-c8e5eca3117d admin admin] PATH_INFO: `/v3/users/fb594a081d194e709023dc7c557afe4c` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:15:58.053702 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75000915-57bf-498d-a557-c8e5eca3117d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb594a081d194e709023dc7c557afe4c)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:15:58.055593 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75000915-57bf-498d-a557-c8e5eca3117d admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:15:58.090953 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:15:58.103441 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-75000915-57bf-498d-a557-c8e5eca3117d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb594a081d194e709023dc7c557afe4c'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:15:58.104092 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-75000915-57bf-498d-a557-c8e5eca3117d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb594a081d194e709023dc7c557afe4c'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:15:58.112704 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-75000915-57bf-498d-a557-c8e5eca3117d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb594a081d194e709023dc7c557afe4c'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:15:58.125130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-75000915-57bf-498d-a557-c8e5eca3117d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb594a081d194e709023dc7c557afe4c'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:15:58.133348 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-75000915-57bf-498d-a557-c8e5eca3117d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb594a081d194e709023dc7c557afe4c'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:15:58.139116 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1085/2162] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:15:57 2022] DELETE /identity/v3/users/fb594a081d194e709023dc7c557afe4c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:15:58.315900 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff2dcd-999d-400b-9613-a4ff7b23e59f service placement] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:15:58.327494 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff2dcd-999d-400b-9613-a4ff7b23e59f service placement] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:15:58.354954 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:15:58.388229 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf5d51-9fb1-45d5-9559-2f3b246edd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:15:58.388881 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf5d51-9fb1-45d5-9559-2f3b246edd61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:15:58.389155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf5d51-9fb1-45d5-9559-2f3b246edd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:15:58.389966 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf5d51-9fb1-45d5-9559-2f3b246edd61 admin admin] PATH_INFO: `/v3/projects/10a83daa0cad4d0fb210247c411e4c0a` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:15:58.391261 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf5d51-9fb1-45d5-9559-2f3b246edd61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10a83daa0cad4d0fb210247c411e4c0a)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:15:58.393172 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf5d51-9fb1-45d5-9559-2f3b246edd61 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:15:58.438332 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-38bf5d51-9fb1-45d5-9559-2f3b246edd61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10a83daa0cad4d0fb210247c411e4c0a', 'request_id': 'req-38bf5d51-9fb1-45d5-9559-2f3b246edd61'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:15:58.444305 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-38bf5d51-9fb1-45d5-9559-2f3b246edd61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10a83daa0cad4d0fb210247c411e4c0a', 'request_id': 'req-38bf5d51-9fb1-45d5-9559-2f3b246edd61'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:15:58.451602 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-38bf5d51-9fb1-45d5-9559-2f3b246edd61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10a83daa0cad4d0fb210247c411e4c0a', 'request_id': 'req-38bf5d51-9fb1-45d5-9559-2f3b246edd61'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:15:58.463454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-38bf5d51-9fb1-45d5-9559-2f3b246edd61 admin admin] The token cache is being invalidate because project 10a83daa0cad4d0fb210247c411e4c0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:15:58.463756 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-38bf5d51-9fb1-45d5-9559-2f3b246edd61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:15:58.464385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-38bf5d51-9fb1-45d5-9559-2f3b246edd61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:15:58.465896 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1078/2163] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:15:58 2022] DELETE /identity/v3/projects/10a83daa0cad4d0fb210247c411e4c0a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:15:58.470117 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75000915-57bf-498d-a557-c8e5eca3117d admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:15:58.471518 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-75000915-57bf-498d-a557-c8e5eca3117d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:15:58.485533 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75000915-57bf-498d-a557-c8e5eca3117d admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:15:58.541212 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb43b04-9f40-40f2-a407-1cc1ea96c905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:15:58.541834 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb43b04-9f40-40f2-a407-1cc1ea96c905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:15:58.542116 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb43b04-9f40-40f2-a407-1cc1ea96c905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:15:58.542392 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb43b04-9f40-40f2-a407-1cc1ea96c905 admin admin] PATH_INFO: `/v3/users/223a5f4ab8914d1c83410fdb2e64390b` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:15:58.543509 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb43b04-9f40-40f2-a407-1cc1ea96c905 admin admin] RBAC: Authorizing `identity:delete_user(user_id=223a5f4ab8914d1c83410fdb2e64390b)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:15:58.545451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb43b04-9f40-40f2-a407-1cc1ea96c905 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:15:58.589944 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-9eb43b04-9f40-40f2-a407-1cc1ea96c905 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '223a5f4ab8914d1c83410fdb2e64390b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:15:58.590593 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-9eb43b04-9f40-40f2-a407-1cc1ea96c905 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '223a5f4ab8914d1c83410fdb2e64390b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:15:58.597677 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-9eb43b04-9f40-40f2-a407-1cc1ea96c905 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '223a5f4ab8914d1c83410fdb2e64390b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:15:58.609713 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-9eb43b04-9f40-40f2-a407-1cc1ea96c905 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '223a5f4ab8914d1c83410fdb2e64390b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:15:58.616807 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-9eb43b04-9f40-40f2-a407-1cc1ea96c905 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '223a5f4ab8914d1c83410fdb2e64390b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:15:58.622786 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1086/2164] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:15:58 2022] DELETE /identity/v3/users/223a5f4ab8914d1c83410fdb2e64390b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:15:58.851692 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf5d51-9fb1-45d5-9559-2f3b246edd61 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:15:58.853236 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-38bf5d51-9fb1-45d5-9559-2f3b246edd61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:15:58.867039 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf5d51-9fb1-45d5-9559-2f3b246edd61 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:15:58.936113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6cd59-1c22-49db-a852-20e05712e446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:15:58.937407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6cd59-1c22-49db-a852-20e05712e446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:15:58.937729 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6cd59-1c22-49db-a852-20e05712e446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:15:58.938007 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6cd59-1c22-49db-a852-20e05712e446 admin admin] PATH_INFO: `/v3/projects/93b00f1b589e4be8b09523fa4cafd09b` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:15:58.939130 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6cd59-1c22-49db-a852-20e05712e446 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93b00f1b589e4be8b09523fa4cafd09b)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:15:58.941118 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6cd59-1c22-49db-a852-20e05712e446 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:15:58.987365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-dae6cd59-1c22-49db-a852-20e05712e446 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93b00f1b589e4be8b09523fa4cafd09b', 'request_id': 'req-dae6cd59-1c22-49db-a852-20e05712e446'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:15:58.993516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-dae6cd59-1c22-49db-a852-20e05712e446 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93b00f1b589e4be8b09523fa4cafd09b', 'request_id': 'req-dae6cd59-1c22-49db-a852-20e05712e446'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:15:58.997266 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:15:59.002956 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-dae6cd59-1c22-49db-a852-20e05712e446 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93b00f1b589e4be8b09523fa4cafd09b', 'request_id': 'req-dae6cd59-1c22-49db-a852-20e05712e446'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:15:59.014592 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-dae6cd59-1c22-49db-a852-20e05712e446 admin admin] The token cache is being invalidate because project 93b00f1b589e4be8b09523fa4cafd09b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:15:59.015011 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-dae6cd59-1c22-49db-a852-20e05712e446 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:15:59.015644 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-dae6cd59-1c22-49db-a852-20e05712e446 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:15:59.017180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1079/2165] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:15:58 2022] DELETE /identity/v3/projects/93b00f1b589e4be8b09523fa4cafd09b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:16:02.487665 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb43b04-9f40-40f2-a407-1cc1ea96c905 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:16:02.489152 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9eb43b04-9f40-40f2-a407-1cc1ea96c905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:16:02.521779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:16:02.548563 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb43b04-9f40-40f2-a407-1cc1ea96c905 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:16:02.595845 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc592c-7010-4806-8458-54240100627b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:16:02.596446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc592c-7010-4806-8458-54240100627b service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:16:02.596775 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc592c-7010-4806-8458-54240100627b service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:16:02.597058 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc592c-7010-4806-8458-54240100627b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:16:02.598780 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-93cc592c-7010-4806-8458-54240100627b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:16:02.607982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc592c-7010-4806-8458-54240100627b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:16:02.609470 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc592c-7010-4806-8458-54240100627b service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:16:02.617324 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1087/2166] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:16:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:16:09.011565 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=4,keystone:DELETE=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:16:11.465002 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6cd59-1c22-49db-a852-20e05712e446 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:16:11.466638 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-dae6cd59-1c22-49db-a852-20e05712e446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:16:11.601996 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6cd59-1c22-49db-a852-20e05712e446 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:16:11.608777 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114a95a-5978-4028-aee9-e7dae40bb02e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:16:11.609506 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114a95a-5978-4028-aee9-e7dae40bb02e service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:16:11.610027 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114a95a-5978-4028-aee9-e7dae40bb02e service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:16:11.610309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114a95a-5978-4028-aee9-e7dae40bb02e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:16:11.612337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b114a95a-5978-4028-aee9-e7dae40bb02e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:16:11.682467 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114a95a-5978-4028-aee9-e7dae40bb02e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:16:11.684100 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114a95a-5978-4028-aee9-e7dae40bb02e service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:16:11.696500 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1080/2167] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:16:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:16:12.613673 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=33 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:16:19.188215 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc592c-7010-4806-8458-54240100627b service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:16:19.189729 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-93cc592c-7010-4806-8458-54240100627b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:16:19.279672 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc592c-7010-4806-8458-54240100627b service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:16:19.284839 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d36aa3-388b-4996-b68e-5cc3ea8adb81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24108b6355414f64ba5e7bf84011eee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:16:19.285444 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d36aa3-388b-4996-b68e-5cc3ea8adb81 service placement] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:16:19.285736 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d36aa3-388b-4996-b68e-5cc3ea8adb81 service placement] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:16:19.286016 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d36aa3-388b-4996-b68e-5cc3ea8adb81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:16:19.287766 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-33d36aa3-388b-4996-b68e-5cc3ea8adb81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:16:19.299897 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d36aa3-388b-4996-b68e-5cc3ea8adb81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:16:19.301504 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d36aa3-388b-4996-b68e-5cc3ea8adb81 service placement] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:16:19.312814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1088/2168] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:16:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:16:21.690189 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=68 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:16:23.118557 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114a95a-5978-4028-aee9-e7dae40bb02e service nova] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:16:23.119980 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b114a95a-5978-4028-aee9-e7dae40bb02e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:16:23.153048 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114a95a-5978-4028-aee9-e7dae40bb02e service nova] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:16:23.156562 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8bdcf-7c38-4600-bec5-4b4b06d2883f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:16:23.157152 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8bdcf-7c38-4600-bec5-4b4b06d2883f service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:16:23.157427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8bdcf-7c38-4600-bec5-4b4b06d2883f service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:16:23.157695 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8bdcf-7c38-4600-bec5-4b4b06d2883f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:16:23.159347 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5fb8bdcf-7c38-4600-bec5-4b4b06d2883f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:16:23.219653 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8bdcf-7c38-4600-bec5-4b4b06d2883f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:16:23.221166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8bdcf-7c38-4600-bec5-4b4b06d2883f service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:16:23.230590 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1081/2169] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:16:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:16:26.380436 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d36aa3-388b-4996-b68e-5cc3ea8adb81 service placement] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:16:26.380436 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-33d36aa3-388b-4996-b68e-5cc3ea8adb81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:16:26.394605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d36aa3-388b-4996-b68e-5cc3ea8adb81 service placement] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:16:26.399334 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abdcc5a-819c-4ed2-b112-149c023eee1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:16:26.400089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdcc5a-819c-4ed2-b112-149c023eee1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:16:26.400467 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdcc5a-819c-4ed2-b112-149c023eee1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:16:26.400855 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdcc5a-819c-4ed2-b112-149c023eee1b admin admin] PATH_INFO: `/v3/users/f95f4fc0d5c5428c96545f543139489b` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:16:26.402069 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abdcc5a-819c-4ed2-b112-149c023eee1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f95f4fc0d5c5428c96545f543139489b)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:16:26.403989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abdcc5a-819c-4ed2-b112-149c023eee1b admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:16:26.451298 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5abdcc5a-819c-4ed2-b112-149c023eee1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f95f4fc0d5c5428c96545f543139489b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:16:26.451942 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5abdcc5a-819c-4ed2-b112-149c023eee1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f95f4fc0d5c5428c96545f543139489b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:16:26.459251 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5abdcc5a-819c-4ed2-b112-149c023eee1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f95f4fc0d5c5428c96545f543139489b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:16:26.471022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5abdcc5a-819c-4ed2-b112-149c023eee1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f95f4fc0d5c5428c96545f543139489b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:16:26.477964 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5abdcc5a-819c-4ed2-b112-149c023eee1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f95f4fc0d5c5428c96545f543139489b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:16:26.484763 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1089/2170] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:16:26 2022] DELETE /identity/v3/users/f95f4fc0d5c5428c96545f543139489b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:16:26.661493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8bdcf-7c38-4600-bec5-4b4b06d2883f service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:16:26.672954 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8bdcf-7c38-4600-bec5-4b4b06d2883f service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:16:26.731117 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66ede2-7413-4c09-ad59-fa59f83575c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:16:26.732014 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66ede2-7413-4c09-ad59-fa59f83575c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:16:26.732301 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66ede2-7413-4c09-ad59-fa59f83575c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:16:26.732614 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66ede2-7413-4c09-ad59-fa59f83575c7 admin admin] PATH_INFO: `/v3/projects/08beca89c7f747229a3cc6cb6a363af4` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:16:26.733788 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66ede2-7413-4c09-ad59-fa59f83575c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08beca89c7f747229a3cc6cb6a363af4)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:16:26.736369 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66ede2-7413-4c09-ad59-fa59f83575c7 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:16:26.782288 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4b66ede2-7413-4c09-ad59-fa59f83575c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08beca89c7f747229a3cc6cb6a363af4', 'request_id': 'req-4b66ede2-7413-4c09-ad59-fa59f83575c7'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:16:26.788224 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4b66ede2-7413-4c09-ad59-fa59f83575c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08beca89c7f747229a3cc6cb6a363af4', 'request_id': 'req-4b66ede2-7413-4c09-ad59-fa59f83575c7'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:16:26.797260 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4b66ede2-7413-4c09-ad59-fa59f83575c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08beca89c7f747229a3cc6cb6a363af4', 'request_id': 'req-4b66ede2-7413-4c09-ad59-fa59f83575c7'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:16:26.807703 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4b66ede2-7413-4c09-ad59-fa59f83575c7 admin admin] The token cache is being invalidate because project 08beca89c7f747229a3cc6cb6a363af4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:16:26.807994 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4b66ede2-7413-4c09-ad59-fa59f83575c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:16:26.808660 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4b66ede2-7413-4c09-ad59-fa59f83575c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:16:26.810127 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1082/2171] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:16:26 2022] DELETE /identity/v3/projects/08beca89c7f747229a3cc6cb6a363af4 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:16:26.815421 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bb7ba-dbbf-49b3-b4ca-2fbd0e48f663 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:16:26.815727 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bb7ba-dbbf-49b3-b4ca-2fbd0e48f663 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:16:26.816008 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bb7ba-dbbf-49b3-b4ca-2fbd0e48f663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:16:26.874129 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-791bb7ba-dbbf-49b3-b4ca-2fbd0e48f663 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:16:26.875867 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-791bb7ba-dbbf-49b3-b4ca-2fbd0e48f663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:16:26.881263 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:16:26.936082 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1090/2172] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:16:26.943803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa5899-50c2-410f-8cd3-24f045eef92f None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:16:26.944099 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa5899-50c2-410f-8cd3-24f045eef92f None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:16:26.944382 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa5899-50c2-410f-8cd3-24f045eef92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:16:26.953770 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:16:26.967169 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-a3fa5899-50c2-410f-8cd3-24f045eef92f None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:16:26.969181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a3fa5899-50c2-410f-8cd3-24f045eef92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:16:26.977187 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1083/2173] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:16:26.983211 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bb7ba-dbbf-49b3-b4ca-2fbd0e48f663 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:16:26.993162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bb7ba-dbbf-49b3-b4ca-2fbd0e48f663 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:16:26.997840 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b5fa8-70ae-4d71-ad2b-862ee9c3b919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:16:26.998430 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b5fa8-70ae-4d71-ad2b-862ee9c3b919 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:16:26.998713 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b5fa8-70ae-4d71-ad2b-862ee9c3b919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:16:26.998993 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b5fa8-70ae-4d71-ad2b-862ee9c3b919 admin admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:16:26.999797 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b5fa8-70ae-4d71-ad2b-862ee9c3b919 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:16:27.000187 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b5fa8-70ae-4d71-ad2b-862ee9c3b919 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:16:27.001855 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b5fa8-70ae-4d71-ad2b-862ee9c3b919 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:16:27.013276 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1091/2174] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:16:27.018226 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa5899-50c2-410f-8cd3-24f045eef92f None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:16:27.027438 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa5899-50c2-410f-8cd3-24f045eef92f None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:16:27.032828 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97dd30-0cdf-456e-957e-8eb538403001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:16:27.033420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97dd30-0cdf-456e-957e-8eb538403001 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:16:27.033701 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97dd30-0cdf-456e-957e-8eb538403001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:16:27.033973 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97dd30-0cdf-456e-957e-8eb538403001 admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:16:27.034600 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97dd30-0cdf-456e-957e-8eb538403001 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:16:27.036471 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97dd30-0cdf-456e-957e-8eb538403001 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:16:27.062757 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1084/2175] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:16:27 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:16:27.066846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b5fa8-70ae-4d71-ad2b-862ee9c3b919 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:16:27.076635 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b5fa8-70ae-4d71-ad2b-862ee9c3b919 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:16:27.133702 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d82a0-0b74-4e41-96e3-bb84530f27f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:16:27.134314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d82a0-0b74-4e41-96e3-bb84530f27f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:16:27.134599 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d82a0-0b74-4e41-96e3-bb84530f27f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:16:27.134900 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d82a0-0b74-4e41-96e3-bb84530f27f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:16:27.135538 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d82a0-0b74-4e41-96e3-bb84530f27f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:16:27.137482 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d82a0-0b74-4e41-96e3-bb84530f27f9 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:16:27.158984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1092/2176] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:16:27 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:16:27.163208 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97dd30-0cdf-456e-957e-8eb538403001 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:16:27.181468 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97dd30-0cdf-456e-957e-8eb538403001 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:16:27.186417 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6887b5-79c8-48b8-ae8f-47440e227646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:16:27.186967 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6887b5-79c8-48b8-ae8f-47440e227646 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:16:27.187361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6887b5-79c8-48b8-ae8f-47440e227646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:16:27.187785 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6887b5-79c8-48b8-ae8f-47440e227646 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:16:27.188465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6887b5-79c8-48b8-ae8f-47440e227646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:16:27.190180 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6887b5-79c8-48b8-ae8f-47440e227646 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:16:27.200307 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1085/2177] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:16:27.205459 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d82a0-0b74-4e41-96e3-bb84530f27f9 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:16:27.215742 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d82a0-0b74-4e41-96e3-bb84530f27f9 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:16:27.221344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec105b6-6eb3-4134-9dba-b8adcf9cc3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:16:27.221937 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec105b6-6eb3-4134-9dba-b8adcf9cc3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:16:27.222224 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec105b6-6eb3-4134-9dba-b8adcf9cc3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:16:27.222529 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec105b6-6eb3-4134-9dba-b8adcf9cc3f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:16:27.223235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec105b6-6eb3-4134-9dba-b8adcf9cc3f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:16:27.224858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec105b6-6eb3-4134-9dba-b8adcf9cc3f2 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:16:27.236908 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1093/2178] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:16:27.241047 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6887b5-79c8-48b8-ae8f-47440e227646 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:16:27.252013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6887b5-79c8-48b8-ae8f-47440e227646 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:16:27.256766 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c0038-1fd4-4d6f-80dc-4d5ae1c34b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:16:27.257407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c0038-1fd4-4d6f-80dc-4d5ae1c34b4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:16:27.257697 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c0038-1fd4-4d6f-80dc-4d5ae1c34b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:16:27.257980 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c0038-1fd4-4d6f-80dc-4d5ae1c34b4c admin admin] PATH_INFO: `/v3/projects/1915ae97801b493fb42f8479cafd9d87/users/06e2b7a78a174a5a82493f58add77988/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:16:27.272573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627c0038-1fd4-4d6f-80dc-4d5ae1c34b4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1915ae97801b493fb42f8479cafd9d87, user_id=06e2b7a78a174a5a82493f58add77988, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:16:27.275184 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627c0038-1fd4-4d6f-80dc-4d5ae1c34b4c admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:16:27.288304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1086/2179] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:16:27 2022] PUT /identity/v3/projects/1915ae97801b493fb42f8479cafd9d87/users/06e2b7a78a174a5a82493f58add77988/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:16:27.293525 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b778f-c472-44fe-9849-468ff89047c0 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:16:27.293822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b778f-c472-44fe-9849-468ff89047c0 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:16:27.294152 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b778f-c472-44fe-9849-468ff89047c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:16:27.314396 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-f05b778f-c472-44fe-9849-468ff89047c0 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:16:27.315581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f05b778f-c472-44fe-9849-468ff89047c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:16:27.377660 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1094/2180] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:16:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:16:27.387441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c0038-1fd4-4d6f-80dc-4d5ae1c34b4c admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:16:27.389222 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-627c0038-1fd4-4d6f-80dc-4d5ae1c34b4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:16:27.449371 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c0038-1fd4-4d6f-80dc-4d5ae1c34b4c admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:16:27.453318 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a7efa8-a77e-434b-ba16-95455c682b76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:16:27.453928 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7efa8-a77e-434b-ba16-95455c682b76 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:16:27.454210 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7efa8-a77e-434b-ba16-95455c682b76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:16:27.454487 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a7efa8-a77e-434b-ba16-95455c682b76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:16:27.465244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a7efa8-a77e-434b-ba16-95455c682b76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:16:27.466672 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a7efa8-a77e-434b-ba16-95455c682b76 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:16:27.477868 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1087/2181] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:16:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:16:31.174438 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6a58d-a71e-40c8-9904-6e03be0fde13 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:16:31.174979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6a58d-a71e-40c8-9904-6e03be0fde13 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:16:31.174979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6a58d-a71e-40c8-9904-6e03be0fde13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:16:31.208114 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-06b6a58d-a71e-40c8-9904-6e03be0fde13 None None] MFA Rules not processed for user `06e2b7a78a174a5a82493f58add77988`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:16:31.209402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-06b6a58d-a71e-40c8-9904-6e03be0fde13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:16:31.214461 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:16:31.277991 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1095/2182] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:16:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:16:31.294138 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a7efa8-a77e-434b-ba16-95455c682b76 service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:16:31.295854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-14a7efa8-a77e-434b-ba16-95455c682b76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:16:31.378628 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a7efa8-a77e-434b-ba16-95455c682b76 service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:16:31.386203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e384bb-2f56-41d8-b70d-01325be8529a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:16:31.386823 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e384bb-2f56-41d8-b70d-01325be8529a service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:16:31.387133 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e384bb-2f56-41d8-b70d-01325be8529a service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:16:31.387408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e384bb-2f56-41d8-b70d-01325be8529a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:16:31.397336 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e384bb-2f56-41d8-b70d-01325be8529a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:16:31.398954 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e384bb-2f56-41d8-b70d-01325be8529a service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:16:31.408739 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1088/2183] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:16:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:16:32.975081 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57309ea-30a2-41e4-a0b6-99cb9df79071 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:16:32.975573 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57309ea-30a2-41e4-a0b6-99cb9df79071 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:16:32.975671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57309ea-30a2-41e4-a0b6-99cb9df79071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:16:32.995693 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-c57309ea-30a2-41e4-a0b6-99cb9df79071 None None] MFA Rules not processed for user `06e2b7a78a174a5a82493f58add77988`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:16:32.996926 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c57309ea-30a2-41e4-a0b6-99cb9df79071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:16:33.004797 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1096/2184] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:16:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:16:41.404098 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=85,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:16:42.990771 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=27 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:16:59.740457 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8db19-f86e-45c6-a720-679091108443 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:16:59.740974 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8db19-f86e-45c6-a720-679091108443 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:16:59.741089 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8db19-f86e-45c6-a720-679091108443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:16:59.761109 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-80e8db19-f86e-45c6-a720-679091108443 None None] MFA Rules not processed for user `f0cf683aa4d14dea974f0febffbbeacc`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:16:59.761896 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-80e8db19-f86e-45c6-a720-679091108443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:16:59.820113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1089/2185] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:16:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:17:07.589634 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978a175-f426-4153-8148-3d0fbadc8468 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:17:07.590206 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978a175-f426-4153-8148-3d0fbadc8468 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:17:07.590489 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978a175-f426-4153-8148-3d0fbadc8468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:17:07.622889 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-5978a175-f426-4153-8148-3d0fbadc8468 None None] MFA Rules not processed for user `7c0356072f92451f98b03a2225e22f86`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:17:07.624078 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-5978a175-f426-4153-8148-3d0fbadc8468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:17:07.678390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1097/2186] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 00:17:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:17:07.727594 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e8db19-f86e-45c6-a720-679091108443 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:17:07.729128 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-80e8db19-f86e-45c6-a720-679091108443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:17:07.811034 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e8db19-f86e-45c6-a720-679091108443 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:17:07.817586 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171d76a-978b-4a43-a58a-90964f7d83da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfd8c7516b7c435d88298672b2596d6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:17:07.818214 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171d76a-978b-4a43-a58a-90964f7d83da service swift] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:17:07.818526 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171d76a-978b-4a43-a58a-90964f7d83da service swift] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:17:07.818806 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171d76a-978b-4a43-a58a-90964f7d83da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:17:07.827727 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c171d76a-978b-4a43-a58a-90964f7d83da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:17:07.829349 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c171d76a-978b-4a43-a58a-90964f7d83da service swift] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:17:07.838452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1090/2187] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 00:17:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:17:17.671230 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=29 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:17:17.834502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=57 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:17:24.453273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c93ca9-3e11-4f74-803c-eea1a1e7d5f7 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:17:24.453832 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c93ca9-3e11-4f74-803c-eea1a1e7d5f7 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:17:24.453937 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c93ca9-3e11-4f74-803c-eea1a1e7d5f7 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:17:24.455256 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1098/2188] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 00:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:17:24.461643 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f906e-00f8-492a-a911-a09e4a4ef57f None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:17:24.462148 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f906e-00f8-492a-a911-a09e4a4ef57f None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:17:24.462330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f906e-00f8-492a-a911-a09e4a4ef57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:17:24.484201 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-d31f906e-00f8-492a-a911-a09e4a4ef57f None None] MFA Rules not processed for user `c4a3579d56294b9d8ad36d4d174c8526`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:17:24.485517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d31f906e-00f8-492a-a911-a09e4a4ef57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:17:24.496625 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1091/2189] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 00:17:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:17:24.507829 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c93ca9-3e11-4f74-803c-eea1a1e7d5f7 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:17:24.509662 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-11c93ca9-3e11-4f74-803c-eea1a1e7d5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:17:24.569073 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c93ca9-3e11-4f74-803c-eea1a1e7d5f7 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:17:24.572920 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8078d-fdec-428f-9903-ca0dbfa61e15 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4a69aeb3fba49b6881f1f15c1758494', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:17:24.573623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8078d-fdec-428f-9903-ca0dbfa61e15 service glance] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:17:24.573975 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8078d-fdec-428f-9903-ca0dbfa61e15 service glance] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:17:24.574318 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8078d-fdec-428f-9903-ca0dbfa61e15 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:17:24.585738 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d8078d-fdec-428f-9903-ca0dbfa61e15 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:17:24.587165 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d8078d-fdec-428f-9903-ca0dbfa61e15 service glance] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:17:24.597920 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1099/2190] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:17:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:17:24.666010 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad00a9d-a300-4d04-bbaf-0e39461755c4 None None] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:17:24.666297 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad00a9d-a300-4d04-bbaf-0e39461755c4 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:17:24.666570 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad00a9d-a300-4d04-bbaf-0e39461755c4 None None] PATH_INFO: `/` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:17:24.667903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1092/2191] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 00:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:17:24.672645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56b1e8-dfe4-4eff-aa53-79b18d5ea9ba None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:17:24.672929 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56b1e8-dfe4-4eff-aa53-79b18d5ea9ba None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:17:24.673200 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56b1e8-dfe4-4eff-aa53-79b18d5ea9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:17:24.692896 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-fa56b1e8-dfe4-4eff-aa53-79b18d5ea9ba None None] MFA Rules not processed for user `a09d45d080bc461c88f38ff281b75b06`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:17:24.694127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-fa56b1e8-dfe4-4eff-aa53-79b18d5ea9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:17:24.700582 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1100/2192] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 00:17:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:17:24.712737 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad00a9d-a300-4d04-bbaf-0e39461755c4 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:17:24.724333 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad00a9d-a300-4d04-bbaf-0e39461755c4 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:17:24.728276 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9cf7b3-33f5-4ca7-9a62-7e63fb5e73f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:17:24.728888 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cf7b3-33f5-4ca7-9a62-7e63fb5e73f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:17:24.729167 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cf7b3-33f5-4ca7-9a62-7e63fb5e73f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:17:24.729438 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cf7b3-33f5-4ca7-9a62-7e63fb5e73f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:17:24.736825 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9cf7b3-33f5-4ca7-9a62-7e63fb5e73f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:17:24.738153 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9cf7b3-33f5-4ca7-9a62-7e63fb5e73f8 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:17:24.746186 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1093/2193] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:17:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:17:24.829111 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2842f9b-329f-46a1-8fb4-b09a6aee1ca0 None None] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:17:24.829401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2842f9b-329f-46a1-8fb4-b09a6aee1ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:17:24.829681 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2842f9b-329f-46a1-8fb4-b09a6aee1ca0 None None] PATH_INFO: `/` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:17:24.831020 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1101/2194] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 00:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:17:24.836887 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23743e24-5213-4eaa-af0a-fc54904b8704 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:17:24.837174 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23743e24-5213-4eaa-af0a-fc54904b8704 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:17:24.837501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23743e24-5213-4eaa-af0a-fc54904b8704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:17:24.857509 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-23743e24-5213-4eaa-af0a-fc54904b8704 None None] MFA Rules not processed for user `c4a3579d56294b9d8ad36d4d174c8526`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:17:24.858951 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-23743e24-5213-4eaa-af0a-fc54904b8704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:17:24.868683 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1094/2195] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 00:17:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:17:24.878932 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2842f9b-329f-46a1-8fb4-b09a6aee1ca0 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:17:24.890031 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2842f9b-329f-46a1-8fb4-b09a6aee1ca0 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:17:24.893609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb4d8a-30b4-41be-b0f7-c48d5d24596d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:17:24.894196 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb4d8a-30b4-41be-b0f7-c48d5d24596d service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:17:24.894476 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb4d8a-30b4-41be-b0f7-c48d5d24596d service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:17:24.894751 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb4d8a-30b4-41be-b0f7-c48d5d24596d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:17:24.905295 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb4d8a-30b4-41be-b0f7-c48d5d24596d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:17:24.906705 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb4d8a-30b4-41be-b0f7-c48d5d24596d service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:17:24.917956 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1102/2196] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:17:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:17:26.655252 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396a492-153d-4b9e-a855-eba18d24d3af None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:17:26.655682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396a492-153d-4b9e-a855-eba18d24d3af None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:17:26.655796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396a492-153d-4b9e-a855-eba18d24d3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:17:26.675420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-c396a492-153d-4b9e-a855-eba18d24d3af None None] MFA Rules not processed for user `7c0356072f92451f98b03a2225e22f86`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:17:26.676684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-c396a492-153d-4b9e-a855-eba18d24d3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:17:26.684288 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1095/2197] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 00:17:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:17:26.697307 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb4d8a-30b4-41be-b0f7-c48d5d24596d service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:17:26.708900 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb4d8a-30b4-41be-b0f7-c48d5d24596d service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:17:26.714294 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afdb4fb-40a7-4cde-8e64-8a473e334599 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfd8c7516b7c435d88298672b2596d6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:17:26.714894 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afdb4fb-40a7-4cde-8e64-8a473e334599 service swift] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:17:26.715177 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afdb4fb-40a7-4cde-8e64-8a473e334599 service swift] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:17:26.715456 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afdb4fb-40a7-4cde-8e64-8a473e334599 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:17:26.723781 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afdb4fb-40a7-4cde-8e64-8a473e334599 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:17:26.725571 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afdb4fb-40a7-4cde-8e64-8a473e334599 service swift] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:17:26.733995 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1103/2198] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 00:17:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:17:36.670628 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=15 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:17:36.730304 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=41 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:17:47.012838 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16587b5a-024e-41d7-9c5b-5ebe2db16b08 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:17:47.013370 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16587b5a-024e-41d7-9c5b-5ebe2db16b08 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:17:47.013370 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16587b5a-024e-41d7-9c5b-5ebe2db16b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:17:47.036951 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-16587b5a-024e-41d7-9c5b-5ebe2db16b08 None None] MFA Rules not processed for user `7c0356072f92451f98b03a2225e22f86`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:17:47.038163 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-16587b5a-024e-41d7-9c5b-5ebe2db16b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:17:47.047687 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1096/2199] 10.222.0.13 () {58 vars in 967 bytes} [Sun Oct 23 00:17:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:17:47.062555 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afdb4fb-40a7-4cde-8e64-8a473e334599 service swift] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:17:47.073316 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afdb4fb-40a7-4cde-8e64-8a473e334599 service swift] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:17:47.078467 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3621409-561f-47f3-b93b-9fd1548fa0f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfd8c7516b7c435d88298672b2596d6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:17:47.079076 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3621409-561f-47f3-b93b-9fd1548fa0f3 service swift] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:17:47.079359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3621409-561f-47f3-b93b-9fd1548fa0f3 service swift] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:17:47.079634 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3621409-561f-47f3-b93b-9fd1548fa0f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:17:47.088995 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3621409-561f-47f3-b93b-9fd1548fa0f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:17:47.090885 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3621409-561f-47f3-b93b-9fd1548fa0f3 service swift] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:17:47.100546 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1104/2200] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 00:17:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:17:57.031387 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:17:57.097090 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=6 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:18:05.655037 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16587b5a-024e-41d7-9c5b-5ebe2db16b08 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:18:05.666310 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16587b5a-024e-41d7-9c5b-5ebe2db16b08 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:18:05.673435 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592d917-b436-45d9-982c-b3fe8fbdf16c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:18:05.674045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592d917-b436-45d9-982c-b3fe8fbdf16c service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:18:05.674326 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592d917-b436-45d9-982c-b3fe8fbdf16c service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:18:05.674597 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592d917-b436-45d9-982c-b3fe8fbdf16c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:18:05.676361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e592d917-b436-45d9-982c-b3fe8fbdf16c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:18:05.691389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592d917-b436-45d9-982c-b3fe8fbdf16c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:18:05.692825 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592d917-b436-45d9-982c-b3fe8fbdf16c service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:18:05.704984 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1097/2201] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:18:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:18:06.891380 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3621409-561f-47f3-b93b-9fd1548fa0f3 service swift] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:18:06.901415 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3621409-561f-47f3-b93b-9fd1548fa0f3 service swift] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:18:06.905028 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e598bf-06e6-4c69-a983-af0be07515c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:18:06.906297 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e598bf-06e6-4c69-a983-af0be07515c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:18:06.906784 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e598bf-06e6-4c69-a983-af0be07515c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:18:06.907065 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e598bf-06e6-4c69-a983-af0be07515c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:18:06.908866 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b6e598bf-06e6-4c69-a983-af0be07515c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:18:06.922671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e598bf-06e6-4c69-a983-af0be07515c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:18:06.924126 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e598bf-06e6-4c69-a983-af0be07515c9 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:18:06.935218 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1105/2202] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:18:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:18:09.432126 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592d917-b436-45d9-982c-b3fe8fbdf16c service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:18:09.433579 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e592d917-b436-45d9-982c-b3fe8fbdf16c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:18:09.444779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592d917-b436-45d9-982c-b3fe8fbdf16c service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:18:09.448602 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fdb781-7e2a-4f36-9580-8fa2275d4d7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:18:09.449203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdb781-7e2a-4f36-9580-8fa2275d4d7c service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:18:09.449489 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdb781-7e2a-4f36-9580-8fa2275d4d7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:18:09.449764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdb781-7e2a-4f36-9580-8fa2275d4d7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:18:09.451499 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e5fdb781-7e2a-4f36-9580-8fa2275d4d7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:18:09.463001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fdb781-7e2a-4f36-9580-8fa2275d4d7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:18:09.464407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fdb781-7e2a-4f36-9580-8fa2275d4d7c service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:18:09.474448 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1098/2203] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:18:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:18:15.690369 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e598bf-06e6-4c69-a983-af0be07515c9 service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:18:15.691786 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b6e598bf-06e6-4c69-a983-af0be07515c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:18:15.772165 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e598bf-06e6-4c69-a983-af0be07515c9 service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:18:15.773126 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fdb781-7e2a-4f36-9580-8fa2275d4d7c service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:18:15.774539 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e5fdb781-7e2a-4f36-9580-8fa2275d4d7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:18:15.787099 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fdb781-7e2a-4f36-9580-8fa2275d4d7c service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:18:15.789561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a902f1-501a-4736-bbe8-53e6fb4dfa22 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24108b6355414f64ba5e7bf84011eee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:18:15.790494 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a902f1-501a-4736-bbe8-53e6fb4dfa22 service placement] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:18:15.790854 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a902f1-501a-4736-bbe8-53e6fb4dfa22 service placement] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:18:15.791205 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a902f1-501a-4736-bbe8-53e6fb4dfa22 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:18:15.793076 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-42a902f1-501a-4736-bbe8-53e6fb4dfa22 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:18:15.795178 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187932b-0492-487c-8535-f12d5d64c074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:18:15.795846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187932b-0492-487c-8535-f12d5d64c074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:18:15.796133 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187932b-0492-487c-8535-f12d5d64c074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:18:15.796411 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187932b-0492-487c-8535-f12d5d64c074 admin admin] PATH_INFO: `/v3/users/cd59419dfebe46a28caad71b1454c53a` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:18:15.800288 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187932b-0492-487c-8535-f12d5d64c074 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd59419dfebe46a28caad71b1454c53a)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:18:15.802563 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187932b-0492-487c-8535-f12d5d64c074 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:18:15.807413 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a902f1-501a-4736-bbe8-53e6fb4dfa22 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:18:15.809120 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a902f1-501a-4736-bbe8-53e6fb4dfa22 service placement] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:18:15.820927 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1106/2204] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:18:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:18:15.861421 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-6187932b-0492-487c-8535-f12d5d64c074 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd59419dfebe46a28caad71b1454c53a'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:18:15.868295 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-6187932b-0492-487c-8535-f12d5d64c074 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd59419dfebe46a28caad71b1454c53a'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:18:15.880635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-6187932b-0492-487c-8535-f12d5d64c074 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd59419dfebe46a28caad71b1454c53a'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:18:15.881311 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-6187932b-0492-487c-8535-f12d5d64c074 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd59419dfebe46a28caad71b1454c53a'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:18:15.881919 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-6187932b-0492-487c-8535-f12d5d64c074 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd59419dfebe46a28caad71b1454c53a'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:18:15.894026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1099/2205] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:18:15 2022] DELETE /identity/v3/users/cd59419dfebe46a28caad71b1454c53a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:18:16.074230 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a902f1-501a-4736-bbe8-53e6fb4dfa22 service placement] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:18:16.088386 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a902f1-501a-4736-bbe8-53e6fb4dfa22 service placement] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:18:16.162265 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e29212-c2c9-478d-91c5-cb7f597bd8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:18:16.163768 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e29212-c2c9-478d-91c5-cb7f597bd8b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:18:16.164352 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e29212-c2c9-478d-91c5-cb7f597bd8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:18:16.164837 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e29212-c2c9-478d-91c5-cb7f597bd8b2 admin admin] PATH_INFO: `/v3/projects/0401b24ace154000a2b9e80e785f1ce9` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:18:16.166177 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e29212-c2c9-478d-91c5-cb7f597bd8b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0401b24ace154000a2b9e80e785f1ce9)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:18:16.168898 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e29212-c2c9-478d-91c5-cb7f597bd8b2 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:18:16.220253 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-42e29212-c2c9-478d-91c5-cb7f597bd8b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0401b24ace154000a2b9e80e785f1ce9', 'request_id': 'req-42e29212-c2c9-478d-91c5-cb7f597bd8b2'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:18:16.227354 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-42e29212-c2c9-478d-91c5-cb7f597bd8b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0401b24ace154000a2b9e80e785f1ce9', 'request_id': 'req-42e29212-c2c9-478d-91c5-cb7f597bd8b2'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:18:16.232907 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-42e29212-c2c9-478d-91c5-cb7f597bd8b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0401b24ace154000a2b9e80e785f1ce9', 'request_id': 'req-42e29212-c2c9-478d-91c5-cb7f597bd8b2'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:18:16.244054 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-42e29212-c2c9-478d-91c5-cb7f597bd8b2 admin admin] The token cache is being invalidate because project 0401b24ace154000a2b9e80e785f1ce9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:18:16.244343 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-42e29212-c2c9-478d-91c5-cb7f597bd8b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:18:16.245018 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-42e29212-c2c9-478d-91c5-cb7f597bd8b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:18:16.246512 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1107/2206] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:18:16 2022] DELETE /identity/v3/projects/0401b24ace154000a2b9e80e785f1ce9 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:18:16.251263 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187932b-0492-487c-8535-f12d5d64c074 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:18:16.252899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-6187932b-0492-487c-8535-f12d5d64c074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:18:16.267097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187932b-0492-487c-8535-f12d5d64c074 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:18:16.330271 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62d7a8-546e-43ce-8258-412a4a3b8110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:18:16.330823 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62d7a8-546e-43ce-8258-412a4a3b8110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:18:16.331107 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62d7a8-546e-43ce-8258-412a4a3b8110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:18:16.331384 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62d7a8-546e-43ce-8258-412a4a3b8110 admin admin] PATH_INFO: `/v3/users/b65519ee09ab4561b8ae537c0f5d363f` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:18:16.332554 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e62d7a8-546e-43ce-8258-412a4a3b8110 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b65519ee09ab4561b8ae537c0f5d363f)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:18:16.334691 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e62d7a8-546e-43ce-8258-412a4a3b8110 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:18:16.380430 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-8e62d7a8-546e-43ce-8258-412a4a3b8110 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b65519ee09ab4561b8ae537c0f5d363f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:18:16.381649 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:18:16.389293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-8e62d7a8-546e-43ce-8258-412a4a3b8110 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b65519ee09ab4561b8ae537c0f5d363f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:18:16.402037 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-8e62d7a8-546e-43ce-8258-412a4a3b8110 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b65519ee09ab4561b8ae537c0f5d363f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:18:16.402706 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-8e62d7a8-546e-43ce-8258-412a4a3b8110 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b65519ee09ab4561b8ae537c0f5d363f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:18:16.403304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-8e62d7a8-546e-43ce-8258-412a4a3b8110 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b65519ee09ab4561b8ae537c0f5d363f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:18:16.423837 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1100/2207] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:18:16 2022] DELETE /identity/v3/users/b65519ee09ab4561b8ae537c0f5d363f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:18:16.594870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e29212-c2c9-478d-91c5-cb7f597bd8b2 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:18:16.596925 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-42e29212-c2c9-478d-91c5-cb7f597bd8b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:18:16.609896 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e29212-c2c9-478d-91c5-cb7f597bd8b2 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:18:16.614146 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:18:16.671392 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f7eab-0fcc-4428-ae28-71d10e2353c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:18:16.672028 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f7eab-0fcc-4428-ae28-71d10e2353c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:18:16.672344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f7eab-0fcc-4428-ae28-71d10e2353c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:18:16.672691 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f7eab-0fcc-4428-ae28-71d10e2353c5 admin admin] PATH_INFO: `/v3/projects/6fe5f4c42da14e2d8d3fbb270d1be923` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:18:16.673880 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057f7eab-0fcc-4428-ae28-71d10e2353c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fe5f4c42da14e2d8d3fbb270d1be923)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:18:16.675783 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057f7eab-0fcc-4428-ae28-71d10e2353c5 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:18:16.722235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-057f7eab-0fcc-4428-ae28-71d10e2353c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fe5f4c42da14e2d8d3fbb270d1be923', 'request_id': 'req-057f7eab-0fcc-4428-ae28-71d10e2353c5'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:18:16.730223 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-057f7eab-0fcc-4428-ae28-71d10e2353c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fe5f4c42da14e2d8d3fbb270d1be923', 'request_id': 'req-057f7eab-0fcc-4428-ae28-71d10e2353c5'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:18:16.736075 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-057f7eab-0fcc-4428-ae28-71d10e2353c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fe5f4c42da14e2d8d3fbb270d1be923', 'request_id': 'req-057f7eab-0fcc-4428-ae28-71d10e2353c5'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:18:16.746873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-057f7eab-0fcc-4428-ae28-71d10e2353c5 admin admin] The token cache is being invalidate because project 6fe5f4c42da14e2d8d3fbb270d1be923 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:18:16.747167 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-057f7eab-0fcc-4428-ae28-71d10e2353c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:18:16.747871 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-057f7eab-0fcc-4428-ae28-71d10e2353c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:18:16.749440 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1108/2208] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:18:16 2022] DELETE /identity/v3/projects/6fe5f4c42da14e2d8d3fbb270d1be923 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:18:26.419355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=12,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:18:26.743922 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:18:31.331998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b7709-82c2-48d3-9676-095e76fb56af None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:18:31.332545 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b7709-82c2-48d3-9676-095e76fb56af None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:18:31.332685 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b7709-82c2-48d3-9676-095e76fb56af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:18:31.352850 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-bd7b7709-82c2-48d3-9676-095e76fb56af None None] MFA Rules not processed for user `06e2b7a78a174a5a82493f58add77988`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:18:31.354048 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-bd7b7709-82c2-48d3-9676-095e76fb56af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:18:31.409427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1101/2209] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:18:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:18:34.014613 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f7eab-0fcc-4428-ae28-71d10e2353c5 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:18:34.016080 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-057f7eab-0fcc-4428-ae28-71d10e2353c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:18:34.074518 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f7eab-0fcc-4428-ae28-71d10e2353c5 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:18:34.078238 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254f5a4-763e-4f80-a449-0f9aab0937b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:18:34.078843 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254f5a4-763e-4f80-a449-0f9aab0937b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:18:34.079128 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254f5a4-763e-4f80-a449-0f9aab0937b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:18:34.079401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254f5a4-763e-4f80-a449-0f9aab0937b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:18:34.082067 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c254f5a4-763e-4f80-a449-0f9aab0937b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:18:34.091516 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c254f5a4-763e-4f80-a449-0f9aab0937b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:18:34.092969 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c254f5a4-763e-4f80-a449-0f9aab0937b4 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:18:34.101141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1109/2210] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:18:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:18:41.402315 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=25 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:18:44.097449 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=26 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:18:55.388829 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b7709-82c2-48d3-9676-095e76fb56af None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:18:55.390487 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-bd7b7709-82c2-48d3-9676-095e76fb56af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:18:55.470128 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b7709-82c2-48d3-9676-095e76fb56af None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:18:55.474442 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4626a-864b-476f-aff9-eda171267bf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:18:55.475044 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4626a-864b-476f-aff9-eda171267bf2 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:18:55.475323 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4626a-864b-476f-aff9-eda171267bf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:18:55.475595 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4626a-864b-476f-aff9-eda171267bf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:18:55.477398 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-01a4626a-864b-476f-aff9-eda171267bf2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:18:55.555797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4626a-864b-476f-aff9-eda171267bf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:18:55.557242 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4626a-864b-476f-aff9-eda171267bf2 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:18:55.575093 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1102/2211] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:18:55 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:19:02.480715 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254f5a4-763e-4f80-a449-0f9aab0937b4 service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:02.490057 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254f5a4-763e-4f80-a449-0f9aab0937b4 service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:02.493616 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2302f81-4972-4de0-ba5f-8038341536af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:02.494206 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2302f81-4972-4de0-ba5f-8038341536af service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:02.494483 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2302f81-4972-4de0-ba5f-8038341536af service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:02.494758 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2302f81-4972-4de0-ba5f-8038341536af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:02.496458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f2302f81-4972-4de0-ba5f-8038341536af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:02.563882 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2302f81-4972-4de0-ba5f-8038341536af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:02.565334 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2302f81-4972-4de0-ba5f-8038341536af service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:02.575540 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1110/2212] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:19:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:19:05.572743 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=70 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:19:05.900675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4626a-864b-476f-aff9-eda171267bf2 service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:05.902118 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-01a4626a-864b-476f-aff9-eda171267bf2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:05.916682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4626a-864b-476f-aff9-eda171267bf2 service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:05.921212 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e21d32-7096-4566-8a8b-1e0a28a73cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:05.921836 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e21d32-7096-4566-8a8b-1e0a28a73cfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:05.922119 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e21d32-7096-4566-8a8b-1e0a28a73cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:05.922398 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e21d32-7096-4566-8a8b-1e0a28a73cfd admin admin] PATH_INFO: `/v3/users/8b4ef165353f4f1189400c005519c2ac` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:05.923512 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e21d32-7096-4566-8a8b-1e0a28a73cfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b4ef165353f4f1189400c005519c2ac)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:05.925437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e21d32-7096-4566-8a8b-1e0a28a73cfd admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:05.971574 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-62e21d32-7096-4566-8a8b-1e0a28a73cfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b4ef165353f4f1189400c005519c2ac'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:05.979244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-62e21d32-7096-4566-8a8b-1e0a28a73cfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b4ef165353f4f1189400c005519c2ac'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:05.991284 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-62e21d32-7096-4566-8a8b-1e0a28a73cfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b4ef165353f4f1189400c005519c2ac'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:05.991944 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-62e21d32-7096-4566-8a8b-1e0a28a73cfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b4ef165353f4f1189400c005519c2ac'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:05.992577 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-62e21d32-7096-4566-8a8b-1e0a28a73cfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b4ef165353f4f1189400c005519c2ac'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:06.005101 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1103/2213] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:19:05 2022] DELETE /identity/v3/users/8b4ef165353f4f1189400c005519c2ac => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:19:06.179710 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2302f81-4972-4de0-ba5f-8038341536af service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:06.189669 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2302f81-4972-4de0-ba5f-8038341536af service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:06.247638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de908714-751b-411a-8726-d2fbd18b8d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:06.248275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de908714-751b-411a-8726-d2fbd18b8d6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:06.248605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de908714-751b-411a-8726-d2fbd18b8d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:06.248884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de908714-751b-411a-8726-d2fbd18b8d6e admin admin] PATH_INFO: `/v3/projects/55445c398b424025b2588386af037755` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:06.250014 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de908714-751b-411a-8726-d2fbd18b8d6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=55445c398b424025b2588386af037755)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:06.251933 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de908714-751b-411a-8726-d2fbd18b8d6e admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:06.303491 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-de908714-751b-411a-8726-d2fbd18b8d6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55445c398b424025b2588386af037755', 'request_id': 'req-de908714-751b-411a-8726-d2fbd18b8d6e'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:06.311569 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-de908714-751b-411a-8726-d2fbd18b8d6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55445c398b424025b2588386af037755', 'request_id': 'req-de908714-751b-411a-8726-d2fbd18b8d6e'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:06.319076 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-de908714-751b-411a-8726-d2fbd18b8d6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55445c398b424025b2588386af037755', 'request_id': 'req-de908714-751b-411a-8726-d2fbd18b8d6e'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:06.332133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-de908714-751b-411a-8726-d2fbd18b8d6e admin admin] The token cache is being invalidate because project 55445c398b424025b2588386af037755 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:19:06.333134 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-de908714-751b-411a-8726-d2fbd18b8d6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:06.333876 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-de908714-751b-411a-8726-d2fbd18b8d6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:06.335595 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1111/2214] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:19:06 2022] DELETE /identity/v3/projects/55445c398b424025b2588386af037755 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:19:06.340728 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c85e2e-6435-4742-8bfc-6ff4b5805a0e None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:06.341024 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c85e2e-6435-4742-8bfc-6ff4b5805a0e None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:06.341308 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c85e2e-6435-4742-8bfc-6ff4b5805a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:06.368732 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-e9c85e2e-6435-4742-8bfc-6ff4b5805a0e None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:19:06.368732 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e9c85e2e-6435-4742-8bfc-6ff4b5805a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:06.432914 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1104/2215] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:19:06.442490 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6983b9-022b-461a-b952-f824f58e138e None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:06.442490 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6983b9-022b-461a-b952-f824f58e138e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:06.442884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6983b9-022b-461a-b952-f824f58e138e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:06.463890 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-ba6983b9-022b-461a-b952-f824f58e138e None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:19:06.465227 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ba6983b9-022b-461a-b952-f824f58e138e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:06.473352 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1112/2216] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:19:06.477649 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c85e2e-6435-4742-8bfc-6ff4b5805a0e None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:06.487550 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c85e2e-6435-4742-8bfc-6ff4b5805a0e None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:06.492761 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d3700-4773-4ae8-9281-73f8608400b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:06.493357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d3700-4773-4ae8-9281-73f8608400b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:06.493652 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d3700-4773-4ae8-9281-73f8608400b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:06.493939 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d3700-4773-4ae8-9281-73f8608400b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:06.494582 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d3700-4773-4ae8-9281-73f8608400b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:19:06.494887 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d3700-4773-4ae8-9281-73f8608400b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:06.496460 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d3700-4773-4ae8-9281-73f8608400b2 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:06.513188 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1105/2217] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:19:06.514416 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6983b9-022b-461a-b952-f824f58e138e None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:06.528179 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6983b9-022b-461a-b952-f824f58e138e None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:06.533136 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d307e7-2d2e-4084-b763-a306e9ac3db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:06.533811 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d307e7-2d2e-4084-b763-a306e9ac3db3 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:06.534093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d307e7-2d2e-4084-b763-a306e9ac3db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:06.534365 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d307e7-2d2e-4084-b763-a306e9ac3db3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:06.535010 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d307e7-2d2e-4084-b763-a306e9ac3db3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:06.536951 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d307e7-2d2e-4084-b763-a306e9ac3db3 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:06.553158 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:19:06.580641 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1113/2218] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:19:06 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:19:06.582021 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d3700-4773-4ae8-9281-73f8608400b2 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:06.595806 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d3700-4773-4ae8-9281-73f8608400b2 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:06.674161 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af583066-fa5e-435b-9b2e-224673e93dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:06.675198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af583066-fa5e-435b-9b2e-224673e93dea admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:06.675711 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af583066-fa5e-435b-9b2e-224673e93dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:06.676295 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af583066-fa5e-435b-9b2e-224673e93dea admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:06.677495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af583066-fa5e-435b-9b2e-224673e93dea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:06.679692 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af583066-fa5e-435b-9b2e-224673e93dea admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:06.688167 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:19:06.707898 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1106/2219] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:19:06 2022] POST /identity/v3/users => generated 286 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:19:06.709355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d307e7-2d2e-4084-b763-a306e9ac3db3 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:06.719734 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d307e7-2d2e-4084-b763-a306e9ac3db3 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:06.725184 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a5db1-1938-4270-b03c-05ccb981c021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:06.725784 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a5db1-1938-4270-b03c-05ccb981c021 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:06.726066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a5db1-1938-4270-b03c-05ccb981c021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:06.726344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a5db1-1938-4270-b03c-05ccb981c021 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:06.726985 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a5db1-1938-4270-b03c-05ccb981c021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:06.728599 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a5db1-1938-4270-b03c-05ccb981c021 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:06.738901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1114/2220] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:19:06.743304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af583066-fa5e-435b-9b2e-224673e93dea admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:06.753298 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af583066-fa5e-435b-9b2e-224673e93dea admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:06.758281 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33a803-f3a7-4ffc-81ca-e9a0b59964b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:06.758869 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33a803-f3a7-4ffc-81ca-e9a0b59964b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:06.759152 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33a803-f3a7-4ffc-81ca-e9a0b59964b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:06.759424 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33a803-f3a7-4ffc-81ca-e9a0b59964b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:06.760049 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33a803-f3a7-4ffc-81ca-e9a0b59964b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:06.761646 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33a803-f3a7-4ffc-81ca-e9a0b59964b2 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:06.773143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1107/2221] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:19:06.777172 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a5db1-1938-4270-b03c-05ccb981c021 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:06.787649 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a5db1-1938-4270-b03c-05ccb981c021 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:06.792289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc154bed-7ee5-4d51-99d2-ba1f50587986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:06.792965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc154bed-7ee5-4d51-99d2-ba1f50587986 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:06.793250 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc154bed-7ee5-4d51-99d2-ba1f50587986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:06.793528 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc154bed-7ee5-4d51-99d2-ba1f50587986 admin admin] PATH_INFO: `/v3/projects/c48287b44b52498da1875ce87b63d3bd/users/95c411770572495d95aaca929c3e33cc/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:06.809996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc154bed-7ee5-4d51-99d2-ba1f50587986 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c48287b44b52498da1875ce87b63d3bd, user_id=95c411770572495d95aaca929c3e33cc, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:06.812647 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc154bed-7ee5-4d51-99d2-ba1f50587986 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:06.825059 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1115/2222] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:19:06 2022] PUT /identity/v3/projects/c48287b44b52498da1875ce87b63d3bd/users/95c411770572495d95aaca929c3e33cc/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:19:06.836364 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff97aa6-bd24-43b0-93dd-9a4e1e162ec7 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:06.836683 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff97aa6-bd24-43b0-93dd-9a4e1e162ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:06.836958 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff97aa6-bd24-43b0-93dd-9a4e1e162ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:06.865014 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-aff97aa6-bd24-43b0-93dd-9a4e1e162ec7 None None] MFA Rules not processed for user `95c411770572495d95aaca929c3e33cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:19:06.866198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-aff97aa6-bd24-43b0-93dd-9a4e1e162ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:06.931464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1108/2223] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:19:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:19:06.936197 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc154bed-7ee5-4d51-99d2-ba1f50587986 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:06.998332 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc154bed-7ee5-4d51-99d2-ba1f50587986 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:07.002880 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25f5b4-20bf-476e-88f9-f08ca7122935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:07.003473 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25f5b4-20bf-476e-88f9-f08ca7122935 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:07.003755 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25f5b4-20bf-476e-88f9-f08ca7122935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:07.004028 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a25f5b4-20bf-476e-88f9-f08ca7122935 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:07.005684 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a25f5b4-20bf-476e-88f9-f08ca7122935 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:07.007849 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a25f5b4-20bf-476e-88f9-f08ca7122935 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:07.034284 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1116/2224] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:19:06 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:19:07.038840 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff97aa6-bd24-43b0-93dd-9a4e1e162ec7 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:07.049295 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff97aa6-bd24-43b0-93dd-9a4e1e162ec7 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:07.113344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e759b-9c44-434f-a638-3aa7172588e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:07.114624 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e759b-9c44-434f-a638-3aa7172588e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:07.114903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e759b-9c44-434f-a638-3aa7172588e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:07.115177 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e759b-9c44-434f-a638-3aa7172588e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:07.115806 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5e759b-9c44-434f-a638-3aa7172588e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:07.117740 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5e759b-9c44-434f-a638-3aa7172588e9 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:07.142809 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1109/2225] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:19:07 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:19:07.147574 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25f5b4-20bf-476e-88f9-f08ca7122935 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:07.158432 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a25f5b4-20bf-476e-88f9-f08ca7122935 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:07.163164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750bb5b3-0857-4baa-97a0-afd1c0f20ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:07.163759 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bb5b3-0857-4baa-97a0-afd1c0f20ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:07.164040 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bb5b3-0857-4baa-97a0-afd1c0f20ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:07.164316 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750bb5b3-0857-4baa-97a0-afd1c0f20ee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:07.165019 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750bb5b3-0857-4baa-97a0-afd1c0f20ee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:07.166600 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750bb5b3-0857-4baa-97a0-afd1c0f20ee0 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:07.177382 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1117/2226] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:19:07.181541 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e759b-9c44-434f-a638-3aa7172588e9 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:07.191539 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e759b-9c44-434f-a638-3aa7172588e9 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:07.196985 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f84a18-9023-491c-b9fc-a7c9181d23fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:07.197629 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f84a18-9023-491c-b9fc-a7c9181d23fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:07.197906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f84a18-9023-491c-b9fc-a7c9181d23fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:07.198187 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f84a18-9023-491c-b9fc-a7c9181d23fc admin admin] PATH_INFO: `/v3/projects/107446a8e1d543eaa50ac1b877ce4c84/users/3a34d4b0b6c54db8b5e9fb1aca42966e/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:07.213371 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f84a18-9023-491c-b9fc-a7c9181d23fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=107446a8e1d543eaa50ac1b877ce4c84, user_id=3a34d4b0b6c54db8b5e9fb1aca42966e, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:07.216093 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f84a18-9023-491c-b9fc-a7c9181d23fc admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:07.230173 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1110/2227] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:19:07 2022] PUT /identity/v3/projects/107446a8e1d543eaa50ac1b877ce4c84/users/3a34d4b0b6c54db8b5e9fb1aca42966e/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:19:07.234042 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750bb5b3-0857-4baa-97a0-afd1c0f20ee0 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:07.244855 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750bb5b3-0857-4baa-97a0-afd1c0f20ee0 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:07.303722 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1e30f-c369-4ca2-9bf3-8515c636930c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:07.304428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1e30f-c369-4ca2-9bf3-8515c636930c admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:07.304799 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1e30f-c369-4ca2-9bf3-8515c636930c admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:07.305139 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1e30f-c369-4ca2-9bf3-8515c636930c admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:07.305952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1e30f-c369-4ca2-9bf3-8515c636930c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:07.307645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1e30f-c369-4ca2-9bf3-8515c636930c admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:07.318665 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1118/2228] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:19:07.322867 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f84a18-9023-491c-b9fc-a7c9181d23fc admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:07.333742 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f84a18-9023-491c-b9fc-a7c9181d23fc admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:07.338455 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa4191-6e8e-43b8-8d99-d1ce8dd3a6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:07.339094 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa4191-6e8e-43b8-8d99-d1ce8dd3a6fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:07.339379 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa4191-6e8e-43b8-8d99-d1ce8dd3a6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:07.339657 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa4191-6e8e-43b8-8d99-d1ce8dd3a6fa admin admin] PATH_INFO: `/v3/projects/107446a8e1d543eaa50ac1b877ce4c84/users/3a34d4b0b6c54db8b5e9fb1aca42966e/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:07.341781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa4191-6e8e-43b8-8d99-d1ce8dd3a6fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=107446a8e1d543eaa50ac1b877ce4c84, user_id=3a34d4b0b6c54db8b5e9fb1aca42966e, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:07.345643 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa4191-6e8e-43b8-8d99-d1ce8dd3a6fa admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:07.358517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1111/2229] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:19:07 2022] PUT /identity/v3/projects/107446a8e1d543eaa50ac1b877ce4c84/users/3a34d4b0b6c54db8b5e9fb1aca42966e/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:19:07.369232 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3238ad-2c61-4149-835f-4daae687b7e0 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:07.369451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3238ad-2c61-4149-835f-4daae687b7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:07.369732 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3238ad-2c61-4149-835f-4daae687b7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:07.400542 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-5f3238ad-2c61-4149-835f-4daae687b7e0 None None] MFA Rules not processed for user `3a34d4b0b6c54db8b5e9fb1aca42966e`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:19:07.401765 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-5f3238ad-2c61-4149-835f-4daae687b7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:07.411465 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:19:07.470487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1119/2230] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:19:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:19:07.482198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa4191-6e8e-43b8-8d99-d1ce8dd3a6fa admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:07.483978 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b3aa4191-6e8e-43b8-8d99-d1ce8dd3a6fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:07.506513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:19:07.543223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa4191-6e8e-43b8-8d99-d1ce8dd3a6fa admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:07.546945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa02e2-5d8d-42fa-b043-69969e418698 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:07.547547 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa02e2-5d8d-42fa-b043-69969e418698 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:07.547834 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa02e2-5d8d-42fa-b043-69969e418698 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:07.548116 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa02e2-5d8d-42fa-b043-69969e418698 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:07.557000 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa02e2-5d8d-42fa-b043-69969e418698 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:07.558410 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa02e2-5d8d-42fa-b043-69969e418698 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:07.567729 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1112/2231] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:19:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:19:07.650295 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3238ad-2c61-4149-835f-4daae687b7e0 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:07.659726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3238ad-2c61-4149-835f-4daae687b7e0 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:07.664150 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2a02e-9666-4410-82d0-3240feaf5a3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:07.664791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2a02e-9666-4410-82d0-3240feaf5a3c service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:07.665075 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2a02e-9666-4410-82d0-3240feaf5a3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:07.665348 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2a02e-9666-4410-82d0-3240feaf5a3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:07.723952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab2a02e-9666-4410-82d0-3240feaf5a3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:07.725416 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab2a02e-9666-4410-82d0-3240feaf5a3c service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:07.734548 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1120/2232] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:19:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:19:11.080892 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa02e2-5d8d-42fa-b043-69969e418698 service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:11.084058 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5dfa02e2-5d8d-42fa-b043-69969e418698 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:11.175538 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa02e2-5d8d-42fa-b043-69969e418698 service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:11.181340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2041f1b7-62fb-41a9-98ca-93f8476ecbdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:11.181971 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041f1b7-62fb-41a9-98ca-93f8476ecbdd service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:11.182256 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041f1b7-62fb-41a9-98ca-93f8476ecbdd service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:11.182535 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041f1b7-62fb-41a9-98ca-93f8476ecbdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:11.184886 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-2041f1b7-62fb-41a9-98ca-93f8476ecbdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:11.198688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2041f1b7-62fb-41a9-98ca-93f8476ecbdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:11.200443 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2041f1b7-62fb-41a9-98ca-93f8476ecbdd service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:11.213096 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1113/2233] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:19:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:19:17.730281 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=52 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:19:20.407840 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2a02e-9666-4410-82d0-3240feaf5a3c service cinder] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:20.409440 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-dab2a02e-9666-4410-82d0-3240feaf5a3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:20.468749 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2a02e-9666-4410-82d0-3240feaf5a3c service cinder] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:20.472556 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70adb500-ae04-4e7e-8347-653f343efe4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:20.473260 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70adb500-ae04-4e7e-8347-653f343efe4f service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:20.473546 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70adb500-ae04-4e7e-8347-653f343efe4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:20.473822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70adb500-ae04-4e7e-8347-653f343efe4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:20.476204 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-70adb500-ae04-4e7e-8347-653f343efe4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:20.541514 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70adb500-ae04-4e7e-8347-653f343efe4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:20.542938 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70adb500-ae04-4e7e-8347-653f343efe4f service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:20.554280 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1121/2234] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:19:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:19:21.209611 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=51 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:19:23.873047 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2041f1b7-62fb-41a9-98ca-93f8476ecbdd service nova] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:23.874637 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-2041f1b7-62fb-41a9-98ca-93f8476ecbdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:23.889929 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2041f1b7-62fb-41a9-98ca-93f8476ecbdd service nova] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:23.894525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74db0f-20dd-4dfe-9db7-681b14ef1dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:23.895149 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74db0f-20dd-4dfe-9db7-681b14ef1dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:23.895437 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74db0f-20dd-4dfe-9db7-681b14ef1dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:23.895716 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74db0f-20dd-4dfe-9db7-681b14ef1dfd admin admin] PATH_INFO: `/v3/users/f0cf683aa4d14dea974f0febffbbeacc` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:23.896945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74db0f-20dd-4dfe-9db7-681b14ef1dfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0cf683aa4d14dea974f0febffbbeacc)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:23.898835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74db0f-20dd-4dfe-9db7-681b14ef1dfd admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:23.944506 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-de74db0f-20dd-4dfe-9db7-681b14ef1dfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0cf683aa4d14dea974f0febffbbeacc'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:23.951363 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-de74db0f-20dd-4dfe-9db7-681b14ef1dfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0cf683aa4d14dea974f0febffbbeacc'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:23.963820 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-de74db0f-20dd-4dfe-9db7-681b14ef1dfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0cf683aa4d14dea974f0febffbbeacc'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:23.964515 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-de74db0f-20dd-4dfe-9db7-681b14ef1dfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0cf683aa4d14dea974f0febffbbeacc'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:23.965136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-de74db0f-20dd-4dfe-9db7-681b14ef1dfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0cf683aa4d14dea974f0febffbbeacc'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:23.978006 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1114/2235] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:19:23 2022] DELETE /identity/v3/users/f0cf683aa4d14dea974f0febffbbeacc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:19:24.148448 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70adb500-ae04-4e7e-8347-653f343efe4f service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:24.159174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70adb500-ae04-4e7e-8347-653f343efe4f service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:24.217532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a36089-7254-49c6-880a-d8b8073a07d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:24.218163 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a36089-7254-49c6-880a-d8b8073a07d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:24.218444 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a36089-7254-49c6-880a-d8b8073a07d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:24.218717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a36089-7254-49c6-880a-d8b8073a07d7 admin admin] PATH_INFO: `/v3/projects/af0eff3dcaa447a6ad585ffc72f64d78` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:24.219816 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a36089-7254-49c6-880a-d8b8073a07d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af0eff3dcaa447a6ad585ffc72f64d78)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:24.221962 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a36089-7254-49c6-880a-d8b8073a07d7 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:24.271513 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-72a36089-7254-49c6-880a-d8b8073a07d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af0eff3dcaa447a6ad585ffc72f64d78', 'request_id': 'req-72a36089-7254-49c6-880a-d8b8073a07d7'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:24.274247 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:19:24.280838 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-72a36089-7254-49c6-880a-d8b8073a07d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af0eff3dcaa447a6ad585ffc72f64d78', 'request_id': 'req-72a36089-7254-49c6-880a-d8b8073a07d7'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:24.288091 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-72a36089-7254-49c6-880a-d8b8073a07d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af0eff3dcaa447a6ad585ffc72f64d78', 'request_id': 'req-72a36089-7254-49c6-880a-d8b8073a07d7'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:24.299289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-72a36089-7254-49c6-880a-d8b8073a07d7 admin admin] The token cache is being invalidate because project af0eff3dcaa447a6ad585ffc72f64d78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:19:24.299583 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-72a36089-7254-49c6-880a-d8b8073a07d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:24.300247 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-72a36089-7254-49c6-880a-d8b8073a07d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:24.301746 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1122/2236] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:19:24 2022] DELETE /identity/v3/projects/af0eff3dcaa447a6ad585ffc72f64d78 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:19:24.307363 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875cbbcd-a357-4938-8042-71b08f9de91a None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:24.307668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875cbbcd-a357-4938-8042-71b08f9de91a None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:24.307969 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875cbbcd-a357-4938-8042-71b08f9de91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:24.328664 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-875cbbcd-a357-4938-8042-71b08f9de91a None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:19:24.329869 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-875cbbcd-a357-4938-8042-71b08f9de91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:24.391298 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1115/2237] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:19:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:19:24.400130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9c412-cee8-47c4-921e-169f94e99b50 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:24.400417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9c412-cee8-47c4-921e-169f94e99b50 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:24.400739 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9c412-cee8-47c4-921e-169f94e99b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:24.422201 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-dca9c412-cee8-47c4-921e-169f94e99b50 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:19:24.423401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-dca9c412-cee8-47c4-921e-169f94e99b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:24.433521 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1123/2238] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:19:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:19:24.437978 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875cbbcd-a357-4938-8042-71b08f9de91a None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:24.447755 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875cbbcd-a357-4938-8042-71b08f9de91a None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:24.452876 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aec4fd-de8a-4792-bb94-30d1b47d470c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:24.453586 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aec4fd-de8a-4792-bb94-30d1b47d470c admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:24.453937 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aec4fd-de8a-4792-bb94-30d1b47d470c admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:24.454282 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aec4fd-de8a-4792-bb94-30d1b47d470c admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:24.455001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aec4fd-de8a-4792-bb94-30d1b47d470c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:19:24.455376 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aec4fd-de8a-4792-bb94-30d1b47d470c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:24.457164 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aec4fd-de8a-4792-bb94-30d1b47d470c admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:24.471374 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1116/2239] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:19:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:19:24.475790 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9c412-cee8-47c4-921e-169f94e99b50 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:24.485616 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9c412-cee8-47c4-921e-169f94e99b50 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:24.491251 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc919726-263c-4833-977a-03c0d1f12a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:24.491965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc919726-263c-4833-977a-03c0d1f12a98 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:24.492305 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc919726-263c-4833-977a-03c0d1f12a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:24.492680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc919726-263c-4833-977a-03c0d1f12a98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:24.493384 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc919726-263c-4833-977a-03c0d1f12a98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:24.495285 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc919726-263c-4833-977a-03c0d1f12a98 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:24.525990 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1124/2240] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:19:24 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:19:24.529932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aec4fd-de8a-4792-bb94-30d1b47d470c admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:24.542547 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aec4fd-de8a-4792-bb94-30d1b47d470c admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:24.620056 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6f53b-8970-4d5b-a99b-3c3fe30df800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:24.620816 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6f53b-8970-4d5b-a99b-3c3fe30df800 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:24.621109 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6f53b-8970-4d5b-a99b-3c3fe30df800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:24.621388 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6f53b-8970-4d5b-a99b-3c3fe30df800 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:24.622022 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6f53b-8970-4d5b-a99b-3c3fe30df800 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:24.624054 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6f53b-8970-4d5b-a99b-3c3fe30df800 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:24.633498 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:19:24.653434 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1117/2241] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:19:24 2022] POST /identity/v3/users => generated 282 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:19:24.654882 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc919726-263c-4833-977a-03c0d1f12a98 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:24.665584 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc919726-263c-4833-977a-03c0d1f12a98 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:24.670124 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397d148-5e41-4da4-9ab6-bb17ef361ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:24.670718 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397d148-5e41-4da4-9ab6-bb17ef361ffa admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:24.671000 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397d148-5e41-4da4-9ab6-bb17ef361ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:24.671335 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397d148-5e41-4da4-9ab6-bb17ef361ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:24.671969 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b397d148-5e41-4da4-9ab6-bb17ef361ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:24.673595 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b397d148-5e41-4da4-9ab6-bb17ef361ffa admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:24.683568 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1125/2242] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:19:24.687496 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6f53b-8970-4d5b-a99b-3c3fe30df800 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:24.697841 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6f53b-8970-4d5b-a99b-3c3fe30df800 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:24.702462 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde07bc-d161-4cbc-83be-453ae7f6e803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:24.703146 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde07bc-d161-4cbc-83be-453ae7f6e803 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:24.703424 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde07bc-d161-4cbc-83be-453ae7f6e803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:24.703697 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde07bc-d161-4cbc-83be-453ae7f6e803 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:24.704345 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde07bc-d161-4cbc-83be-453ae7f6e803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:24.705940 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde07bc-d161-4cbc-83be-453ae7f6e803 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:24.717637 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1118/2243] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:19:24.721776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397d148-5e41-4da4-9ab6-bb17ef361ffa admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:24.732167 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397d148-5e41-4da4-9ab6-bb17ef361ffa admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:24.736990 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b4b7d-b827-444c-ac91-61e7afc2cd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:24.737918 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b4b7d-b827-444c-ac91-61e7afc2cd0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:24.738201 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b4b7d-b827-444c-ac91-61e7afc2cd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:24.738481 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b4b7d-b827-444c-ac91-61e7afc2cd0a admin admin] PATH_INFO: `/v3/projects/4d5ad671d1ae45818604cea55efc555b/users/a4ac717ecebc42d28a58986a4bcd1864/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:24.753433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b4b7d-b827-444c-ac91-61e7afc2cd0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d5ad671d1ae45818604cea55efc555b, user_id=a4ac717ecebc42d28a58986a4bcd1864, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:24.756310 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b4b7d-b827-444c-ac91-61e7afc2cd0a admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:24.768965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1126/2244] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:19:24 2022] PUT /identity/v3/projects/4d5ad671d1ae45818604cea55efc555b/users/a4ac717ecebc42d28a58986a4bcd1864/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:19:24.779882 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6ad38-912c-41b7-9987-51c6471fe75a None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:24.780178 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6ad38-912c-41b7-9987-51c6471fe75a None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:24.780460 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6ad38-912c-41b7-9987-51c6471fe75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:24.809717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-13d6ad38-912c-41b7-9987-51c6471fe75a None None] MFA Rules not processed for user `a4ac717ecebc42d28a58986a4bcd1864`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:19:24.810931 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-13d6ad38-912c-41b7-9987-51c6471fe75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:24.875140 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1119/2245] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:19:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:19:24.879802 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b4b7d-b827-444c-ac91-61e7afc2cd0a admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:24.940542 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b4b7d-b827-444c-ac91-61e7afc2cd0a admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:24.946057 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b27c780-4cad-4972-9670-1eb0dd3526ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:24.946651 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27c780-4cad-4972-9670-1eb0dd3526ae admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:24.946956 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27c780-4cad-4972-9670-1eb0dd3526ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:24.947232 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27c780-4cad-4972-9670-1eb0dd3526ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:24.947846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b27c780-4cad-4972-9670-1eb0dd3526ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:24.950860 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b27c780-4cad-4972-9670-1eb0dd3526ae admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:24.976761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1127/2246] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:19:24 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:19:24.980735 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6ad38-912c-41b7-9987-51c6471fe75a None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:24.991668 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6ad38-912c-41b7-9987-51c6471fe75a None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:25.047262 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea48dc-d657-48c7-a412-c73ab7140661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:25.047867 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea48dc-d657-48c7-a412-c73ab7140661 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:25.048147 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea48dc-d657-48c7-a412-c73ab7140661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:25.048422 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea48dc-d657-48c7-a412-c73ab7140661 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:25.049088 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea48dc-d657-48c7-a412-c73ab7140661 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:25.050981 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea48dc-d657-48c7-a412-c73ab7140661 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:25.074091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1120/2247] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:19:24 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:19:25.077358 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b27c780-4cad-4972-9670-1eb0dd3526ae admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:25.087760 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b27c780-4cad-4972-9670-1eb0dd3526ae admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:25.092445 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a47e35-e358-4023-9b67-d4eaf7dac0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:25.093070 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a47e35-e358-4023-9b67-d4eaf7dac0d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:25.093358 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a47e35-e358-4023-9b67-d4eaf7dac0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:25.093639 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a47e35-e358-4023-9b67-d4eaf7dac0d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:25.094280 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a47e35-e358-4023-9b67-d4eaf7dac0d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:25.095930 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a47e35-e358-4023-9b67-d4eaf7dac0d8 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:25.107671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1128/2248] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:19:25.111498 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea48dc-d657-48c7-a412-c73ab7140661 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:25.122773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea48dc-d657-48c7-a412-c73ab7140661 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:25.127524 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d21755-e0c7-4774-9177-6666157470ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:25.128175 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d21755-e0c7-4774-9177-6666157470ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:25.128464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d21755-e0c7-4774-9177-6666157470ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:25.128775 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d21755-e0c7-4774-9177-6666157470ac admin admin] PATH_INFO: `/v3/projects/2b3b30b989c14a0b98cb75e8b4c12fc7/users/f074c035a65549a1be61160d894e7cf0/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:25.143505 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d21755-e0c7-4774-9177-6666157470ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b3b30b989c14a0b98cb75e8b4c12fc7, user_id=f074c035a65549a1be61160d894e7cf0, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:25.146241 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d21755-e0c7-4774-9177-6666157470ac admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:25.158860 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1121/2249] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:19:25 2022] PUT /identity/v3/projects/2b3b30b989c14a0b98cb75e8b4c12fc7/users/f074c035a65549a1be61160d894e7cf0/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:19:25.163658 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a47e35-e358-4023-9b67-d4eaf7dac0d8 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:25.173737 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a47e35-e358-4023-9b67-d4eaf7dac0d8 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:25.221391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:19:25.233602 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c8a56-c5b6-44d1-bd12-98fe0b9ba8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:25.234199 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c8a56-c5b6-44d1-bd12-98fe0b9ba8fe admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:25.234476 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c8a56-c5b6-44d1-bd12-98fe0b9ba8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:25.234747 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c8a56-c5b6-44d1-bd12-98fe0b9ba8fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:25.235377 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c8a56-c5b6-44d1-bd12-98fe0b9ba8fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:25.237008 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c8a56-c5b6-44d1-bd12-98fe0b9ba8fe admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:25.247868 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1129/2250] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:19:25.252077 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d21755-e0c7-4774-9177-6666157470ac admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:25.262809 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d21755-e0c7-4774-9177-6666157470ac admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:25.267714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390b89f-6f9b-4492-86db-adf0f65bdfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:25.268361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390b89f-6f9b-4492-86db-adf0f65bdfdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:25.268681 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390b89f-6f9b-4492-86db-adf0f65bdfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:25.268964 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d390b89f-6f9b-4492-86db-adf0f65bdfdb admin admin] PATH_INFO: `/v3/projects/2b3b30b989c14a0b98cb75e8b4c12fc7/users/f074c035a65549a1be61160d894e7cf0/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:25.270887 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390b89f-6f9b-4492-86db-adf0f65bdfdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b3b30b989c14a0b98cb75e8b4c12fc7, user_id=f074c035a65549a1be61160d894e7cf0, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:25.274332 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d390b89f-6f9b-4492-86db-adf0f65bdfdb admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:25.284936 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1122/2251] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:19:25 2022] PUT /identity/v3/projects/2b3b30b989c14a0b98cb75e8b4c12fc7/users/f074c035a65549a1be61160d894e7cf0/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:19:25.297633 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ba096-abfa-40ca-b8ed-af8f44a365bb None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:25.297964 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ba096-abfa-40ca-b8ed-af8f44a365bb None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:25.298244 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ba096-abfa-40ca-b8ed-af8f44a365bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:25.326072 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-650ba096-abfa-40ca-b8ed-af8f44a365bb None None] MFA Rules not processed for user `f074c035a65549a1be61160d894e7cf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:19:25.327371 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-650ba096-abfa-40ca-b8ed-af8f44a365bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:25.394809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1130/2252] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:19:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:19:25.407686 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390b89f-6f9b-4492-86db-adf0f65bdfdb admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:25.409449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d390b89f-6f9b-4492-86db-adf0f65bdfdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:25.432279 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:19:25.470373 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d390b89f-6f9b-4492-86db-adf0f65bdfdb admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:25.474024 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7c13b-a346-4387-b29b-0168cffce709 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:25.475252 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7c13b-a346-4387-b29b-0168cffce709 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:25.475540 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7c13b-a346-4387-b29b-0168cffce709 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:25.475811 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7c13b-a346-4387-b29b-0168cffce709 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:25.485314 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7c13b-a346-4387-b29b-0168cffce709 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:25.486714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7c13b-a346-4387-b29b-0168cffce709 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:25.496755 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1123/2253] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:19:25 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:19:25.572793 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ba096-abfa-40ca-b8ed-af8f44a365bb None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:25.582528 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ba096-abfa-40ca-b8ed-af8f44a365bb None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:25.586666 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901ee42-c07d-4f76-ba05-6332413fe96e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:25.587266 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901ee42-c07d-4f76-ba05-6332413fe96e service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:25.587555 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901ee42-c07d-4f76-ba05-6332413fe96e service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:25.587833 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901ee42-c07d-4f76-ba05-6332413fe96e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:25.644946 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901ee42-c07d-4f76-ba05-6332413fe96e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:25.646868 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901ee42-c07d-4f76-ba05-6332413fe96e service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:25.657081 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1131/2254] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:19:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:19:28.415062 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7c13b-a346-4387-b29b-0168cffce709 service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:28.477616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7c13b-a346-4387-b29b-0168cffce709 service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:28.482139 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e199669-df29-45d6-8e5d-181ff271651f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:28.482760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e199669-df29-45d6-8e5d-181ff271651f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:28.483070 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e199669-df29-45d6-8e5d-181ff271651f admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:28.483347 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e199669-df29-45d6-8e5d-181ff271651f admin admin] PATH_INFO: `/v3/users/a4ac717ecebc42d28a58986a4bcd1864` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:28.484460 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e199669-df29-45d6-8e5d-181ff271651f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4ac717ecebc42d28a58986a4bcd1864)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:28.486376 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e199669-df29-45d6-8e5d-181ff271651f admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:28.531722 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1e199669-df29-45d6-8e5d-181ff271651f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4ac717ecebc42d28a58986a4bcd1864'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:28.538643 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1e199669-df29-45d6-8e5d-181ff271651f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4ac717ecebc42d28a58986a4bcd1864'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:28.550479 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1e199669-df29-45d6-8e5d-181ff271651f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4ac717ecebc42d28a58986a4bcd1864'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:28.551094 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1e199669-df29-45d6-8e5d-181ff271651f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4ac717ecebc42d28a58986a4bcd1864'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:28.551704 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1e199669-df29-45d6-8e5d-181ff271651f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4ac717ecebc42d28a58986a4bcd1864'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:28.564648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1124/2255] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:19:28 2022] DELETE /identity/v3/users/a4ac717ecebc42d28a58986a4bcd1864 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:19:28.569141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4a047-27d5-4e9e-a675-9412f502f45a None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:28.569633 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4a047-27d5-4e9e-a675-9412f502f45a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:28.569784 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4a047-27d5-4e9e-a675-9412f502f45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:28.589498 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-58d4a047-27d5-4e9e-a675-9412f502f45a None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:19:28.590706 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-58d4a047-27d5-4e9e-a675-9412f502f45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:28.649600 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1132/2256] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:19:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:19:28.659388 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e199669-df29-45d6-8e5d-181ff271651f admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:28.660873 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1e199669-df29-45d6-8e5d-181ff271651f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:28.706728 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:19:28.721493 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e199669-df29-45d6-8e5d-181ff271651f admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:28.725343 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81621c6-2dd2-4310-8559-03d09fd5494e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:28.725947 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81621c6-2dd2-4310-8559-03d09fd5494e service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:28.726228 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81621c6-2dd2-4310-8559-03d09fd5494e service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:28.726500 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81621c6-2dd2-4310-8559-03d09fd5494e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:28.736256 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81621c6-2dd2-4310-8559-03d09fd5494e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:28.737955 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81621c6-2dd2-4310-8559-03d09fd5494e service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:28.748042 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1125/2257] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:19:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:19:29.016790 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4a047-27d5-4e9e-a675-9412f502f45a None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:29.018320 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-58d4a047-27d5-4e9e-a675-9412f502f45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:29.031365 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d4a047-27d5-4e9e-a675-9412f502f45a None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:29.036074 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be73988-48e9-43af-b3ba-3329c8bae076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:29.036818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be73988-48e9-43af-b3ba-3329c8bae076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:29.037172 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be73988-48e9-43af-b3ba-3329c8bae076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:29.037519 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be73988-48e9-43af-b3ba-3329c8bae076 admin admin] PATH_INFO: `/v3/projects/4d5ad671d1ae45818604cea55efc555b` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:29.038719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be73988-48e9-43af-b3ba-3329c8bae076 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d5ad671d1ae45818604cea55efc555b)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:29.041281 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be73988-48e9-43af-b3ba-3329c8bae076 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:29.047909 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:19:29.087587 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-3be73988-48e9-43af-b3ba-3329c8bae076 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d5ad671d1ae45818604cea55efc555b', 'request_id': 'req-3be73988-48e9-43af-b3ba-3329c8bae076'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:29.094683 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-3be73988-48e9-43af-b3ba-3329c8bae076 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d5ad671d1ae45818604cea55efc555b', 'request_id': 'req-3be73988-48e9-43af-b3ba-3329c8bae076'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:29.106087 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-3be73988-48e9-43af-b3ba-3329c8bae076 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d5ad671d1ae45818604cea55efc555b', 'request_id': 'req-3be73988-48e9-43af-b3ba-3329c8bae076'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:29.113449 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-3be73988-48e9-43af-b3ba-3329c8bae076 admin admin] The token cache is being invalidate because project 4d5ad671d1ae45818604cea55efc555b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:19:29.113610 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-3be73988-48e9-43af-b3ba-3329c8bae076 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:29.114297 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-3be73988-48e9-43af-b3ba-3329c8bae076 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:29.115795 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1133/2258] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:19:29 2022] DELETE /identity/v3/projects/4d5ad671d1ae45818604cea55efc555b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:19:29.119934 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81621c6-2dd2-4310-8559-03d09fd5494e service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:29.130117 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81621c6-2dd2-4310-8559-03d09fd5494e service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:29.190216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd342039-0968-44a6-b9e3-c6a92a04f45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:29.190850 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd342039-0968-44a6-b9e3-c6a92a04f45d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:29.191139 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd342039-0968-44a6-b9e3-c6a92a04f45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:29.191418 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd342039-0968-44a6-b9e3-c6a92a04f45d admin admin] PATH_INFO: `/v3/users/f074c035a65549a1be61160d894e7cf0` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:29.192663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd342039-0968-44a6-b9e3-c6a92a04f45d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f074c035a65549a1be61160d894e7cf0)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:29.194597 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd342039-0968-44a6-b9e3-c6a92a04f45d admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:29.249940 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-fd342039-0968-44a6-b9e3-c6a92a04f45d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f074c035a65549a1be61160d894e7cf0'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:29.257392 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-fd342039-0968-44a6-b9e3-c6a92a04f45d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f074c035a65549a1be61160d894e7cf0'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:29.269403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-fd342039-0968-44a6-b9e3-c6a92a04f45d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f074c035a65549a1be61160d894e7cf0'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:29.270087 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-fd342039-0968-44a6-b9e3-c6a92a04f45d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f074c035a65549a1be61160d894e7cf0'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:29.270698 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-fd342039-0968-44a6-b9e3-c6a92a04f45d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f074c035a65549a1be61160d894e7cf0'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:29.283714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1126/2259] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:19:29 2022] DELETE /identity/v3/users/f074c035a65549a1be61160d894e7cf0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:19:29.554390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be73988-48e9-43af-b3ba-3329c8bae076 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:29.556164 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-3be73988-48e9-43af-b3ba-3329c8bae076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:29.568815 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be73988-48e9-43af-b3ba-3329c8bae076 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:29.625140 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe8900-fda1-46bc-b3bc-8a1b9f7d9cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:29.625807 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe8900-fda1-46bc-b3bc-8a1b9f7d9cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:29.626032 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe8900-fda1-46bc-b3bc-8a1b9f7d9cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:29.626667 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe8900-fda1-46bc-b3bc-8a1b9f7d9cde admin admin] PATH_INFO: `/v3/projects/2b3b30b989c14a0b98cb75e8b4c12fc7` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:29.627458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe8900-fda1-46bc-b3bc-8a1b9f7d9cde admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b3b30b989c14a0b98cb75e8b4c12fc7)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:29.629414 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe8900-fda1-46bc-b3bc-8a1b9f7d9cde admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:29.675723 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-0abe8900-fda1-46bc-b3bc-8a1b9f7d9cde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b3b30b989c14a0b98cb75e8b4c12fc7', 'request_id': 'req-0abe8900-fda1-46bc-b3bc-8a1b9f7d9cde'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:29.682929 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-0abe8900-fda1-46bc-b3bc-8a1b9f7d9cde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b3b30b989c14a0b98cb75e8b4c12fc7', 'request_id': 'req-0abe8900-fda1-46bc-b3bc-8a1b9f7d9cde'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:29.688505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-0abe8900-fda1-46bc-b3bc-8a1b9f7d9cde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b3b30b989c14a0b98cb75e8b4c12fc7', 'request_id': 'req-0abe8900-fda1-46bc-b3bc-8a1b9f7d9cde'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:29.699977 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-0abe8900-fda1-46bc-b3bc-8a1b9f7d9cde admin admin] The token cache is being invalidate because project 2b3b30b989c14a0b98cb75e8b4c12fc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:19:29.700268 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-0abe8900-fda1-46bc-b3bc-8a1b9f7d9cde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:29.700917 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-0abe8900-fda1-46bc-b3bc-8a1b9f7d9cde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:29.702516 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1134/2260] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:19:29 2022] DELETE /identity/v3/projects/2b3b30b989c14a0b98cb75e8b4c12fc7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:19:29.707549 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112b836-e983-4e35-a5e5-6746893742e6 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:29.708005 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112b836-e983-4e35-a5e5-6746893742e6 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:29.708101 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112b836-e983-4e35-a5e5-6746893742e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:29.728177 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1112b836-e983-4e35-a5e5-6746893742e6 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:19:29.729421 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1112b836-e983-4e35-a5e5-6746893742e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:29.787881 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1127/2261] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:19:29.795696 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f5ed5-89ee-4634-be3e-7405c14bd0d5 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:29.795987 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f5ed5-89ee-4634-be3e-7405c14bd0d5 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:29.796287 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f5ed5-89ee-4634-be3e-7405c14bd0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:29.816212 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-791f5ed5-89ee-4634-be3e-7405c14bd0d5 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:19:29.817447 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-791f5ed5-89ee-4634-be3e-7405c14bd0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:29.826054 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1135/2262] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:19:29.830419 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112b836-e983-4e35-a5e5-6746893742e6 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:29.836594 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:19:29.840609 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112b836-e983-4e35-a5e5-6746893742e6 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:29.846930 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c308be-4eb8-4eb4-b924-849c35f17ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:29.847609 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c308be-4eb8-4eb4-b924-849c35f17ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:29.848090 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c308be-4eb8-4eb4-b924-849c35f17ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:29.848661 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c308be-4eb8-4eb4-b924-849c35f17ca0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:29.849484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c308be-4eb8-4eb4-b924-849c35f17ca0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:19:29.849801 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c308be-4eb8-4eb4-b924-849c35f17ca0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:29.851373 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c308be-4eb8-4eb4-b924-849c35f17ca0 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:29.863225 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1128/2263] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:19:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:19:29.867612 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f5ed5-89ee-4634-be3e-7405c14bd0d5 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:29.877293 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f5ed5-89ee-4634-be3e-7405c14bd0d5 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:29.881998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698e607-28a6-435b-a26f-63f9ce1f696c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:29.882532 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698e607-28a6-435b-a26f-63f9ce1f696c admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:29.882816 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698e607-28a6-435b-a26f-63f9ce1f696c admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:29.883094 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698e607-28a6-435b-a26f-63f9ce1f696c admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:29.883901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698e607-28a6-435b-a26f-63f9ce1f696c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:29.885809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698e607-28a6-435b-a26f-63f9ce1f696c admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:29.901746 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:19:29.915326 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1136/2264] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:19:29 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:19:29.920097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c308be-4eb8-4eb4-b924-849c35f17ca0 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:29.930249 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c308be-4eb8-4eb4-b924-849c35f17ca0 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:29.987281 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e10c6-d023-4796-bbd0-483d8bdaeaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:29.988128 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e10c6-d023-4796-bbd0-483d8bdaeaa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:29.988412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e10c6-d023-4796-bbd0-483d8bdaeaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:29.988731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e10c6-d023-4796-bbd0-483d8bdaeaa5 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:29.989346 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e10c6-d023-4796-bbd0-483d8bdaeaa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:29.991273 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4e10c6-d023-4796-bbd0-483d8bdaeaa5 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:30.012192 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1129/2265] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:19:29 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:19:30.016437 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698e607-28a6-435b-a26f-63f9ce1f696c admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:30.027219 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698e607-28a6-435b-a26f-63f9ce1f696c admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:30.031935 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87377419-b072-4190-9185-21be5dd6f474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:30.032546 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87377419-b072-4190-9185-21be5dd6f474 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:30.032842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87377419-b072-4190-9185-21be5dd6f474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:30.033112 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87377419-b072-4190-9185-21be5dd6f474 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:30.033746 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87377419-b072-4190-9185-21be5dd6f474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:30.035334 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87377419-b072-4190-9185-21be5dd6f474 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:30.045880 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1137/2266] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:19:30.049911 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e10c6-d023-4796-bbd0-483d8bdaeaa5 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:30.059899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e10c6-d023-4796-bbd0-483d8bdaeaa5 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:30.064839 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458905f-5a6a-4e90-94b4-ee182a5040d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:30.065426 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458905f-5a6a-4e90-94b4-ee182a5040d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:30.065728 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458905f-5a6a-4e90-94b4-ee182a5040d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:30.065985 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458905f-5a6a-4e90-94b4-ee182a5040d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:30.066616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0458905f-5a6a-4e90-94b4-ee182a5040d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:30.068188 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0458905f-5a6a-4e90-94b4-ee182a5040d7 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:30.079229 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1130/2267] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:19:30.083305 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87377419-b072-4190-9185-21be5dd6f474 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:30.093969 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87377419-b072-4190-9185-21be5dd6f474 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:30.098530 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd544ad-430f-4f48-8ca0-8d89d37eb646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:30.099176 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd544ad-430f-4f48-8ca0-8d89d37eb646 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:30.099462 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd544ad-430f-4f48-8ca0-8d89d37eb646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:30.099746 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd544ad-430f-4f48-8ca0-8d89d37eb646 admin admin] PATH_INFO: `/v3/projects/342b9acebd1246d2bf62664e062fa4d7/users/aafd282958b14ae1800408bc8621e870/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:30.115862 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd544ad-430f-4f48-8ca0-8d89d37eb646 admin admin] RBAC: Authorizing `identity:create_grant(project_id=342b9acebd1246d2bf62664e062fa4d7, user_id=aafd282958b14ae1800408bc8621e870, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:30.118580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd544ad-430f-4f48-8ca0-8d89d37eb646 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:30.131690 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1138/2268] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:19:30 2022] PUT /identity/v3/projects/342b9acebd1246d2bf62664e062fa4d7/users/aafd282958b14ae1800408bc8621e870/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:19:30.143766 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad333c5-809e-4b3d-9ce2-a569976abfa0 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:30.143995 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad333c5-809e-4b3d-9ce2-a569976abfa0 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:30.144329 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad333c5-809e-4b3d-9ce2-a569976abfa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:30.172510 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-aad333c5-809e-4b3d-9ce2-a569976abfa0 None None] MFA Rules not processed for user `aafd282958b14ae1800408bc8621e870`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:19:30.173726 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-aad333c5-809e-4b3d-9ce2-a569976abfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:30.238219 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1131/2269] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:19:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:19:30.242719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd544ad-430f-4f48-8ca0-8d89d37eb646 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:30.307046 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd544ad-430f-4f48-8ca0-8d89d37eb646 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:30.312582 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7dc65-2bf3-4fb2-8c53-d7f1382c17a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:30.313426 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7dc65-2bf3-4fb2-8c53-d7f1382c17a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:30.313859 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7dc65-2bf3-4fb2-8c53-d7f1382c17a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:30.314210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7dc65-2bf3-4fb2-8c53-d7f1382c17a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:30.315066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a7dc65-2bf3-4fb2-8c53-d7f1382c17a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:30.317805 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a7dc65-2bf3-4fb2-8c53-d7f1382c17a1 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:30.350856 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1139/2270] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:19:30 2022] POST /identity/v3/projects => generated 391 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:19:30.352633 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad333c5-809e-4b3d-9ce2-a569976abfa0 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:30.364771 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad333c5-809e-4b3d-9ce2-a569976abfa0 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:30.423351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b5add-646b-4082-bcbd-b36e48c3fd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:30.423961 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b5add-646b-4082-bcbd-b36e48c3fd46 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:30.424244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b5add-646b-4082-bcbd-b36e48c3fd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:30.424562 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b5add-646b-4082-bcbd-b36e48c3fd46 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:30.425211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26b5add-646b-4082-bcbd-b36e48c3fd46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:30.427120 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26b5add-646b-4082-bcbd-b36e48c3fd46 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:30.435748 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:19:30.451381 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1132/2271] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:19:30 2022] POST /identity/v3/users => generated 302 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:19:30.455623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7dc65-2bf3-4fb2-8c53-d7f1382c17a1 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:30.466844 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7dc65-2bf3-4fb2-8c53-d7f1382c17a1 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:30.471605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40acde60-f328-44da-80d2-56f254aa116b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:30.472294 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40acde60-f328-44da-80d2-56f254aa116b admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:30.472676 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40acde60-f328-44da-80d2-56f254aa116b admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:30.473038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40acde60-f328-44da-80d2-56f254aa116b admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:30.473780 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40acde60-f328-44da-80d2-56f254aa116b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:30.475421 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40acde60-f328-44da-80d2-56f254aa116b admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:30.486748 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1140/2272] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:19:30.491300 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b5add-646b-4082-bcbd-b36e48c3fd46 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:30.502340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b5add-646b-4082-bcbd-b36e48c3fd46 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:30.509023 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d591031-15b7-4ffc-abf5-000c66a3666f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:30.509883 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d591031-15b7-4ffc-abf5-000c66a3666f admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:30.510239 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d591031-15b7-4ffc-abf5-000c66a3666f admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:30.510611 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d591031-15b7-4ffc-abf5-000c66a3666f admin admin] PATH_INFO: `/v3/projects/bc486403073d41609f7b00f732c905ec/users/787e37c0cb2e4e37b67145241eb257fb/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:30.529359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d591031-15b7-4ffc-abf5-000c66a3666f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc486403073d41609f7b00f732c905ec, user_id=787e37c0cb2e4e37b67145241eb257fb, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:30.532082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d591031-15b7-4ffc-abf5-000c66a3666f admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:30.546497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1133/2273] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:19:30 2022] PUT /identity/v3/projects/bc486403073d41609f7b00f732c905ec/users/787e37c0cb2e4e37b67145241eb257fb/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:19:30.550952 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40acde60-f328-44da-80d2-56f254aa116b admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:30.566243 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40acde60-f328-44da-80d2-56f254aa116b admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:30.630393 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5673d-abd2-483c-a000-266d72c8e552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:30.631038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5673d-abd2-483c-a000-266d72c8e552 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:30.631318 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5673d-abd2-483c-a000-266d72c8e552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:30.631714 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5673d-abd2-483c-a000-266d72c8e552 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:30.632365 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb5673d-abd2-483c-a000-266d72c8e552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:30.635311 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb5673d-abd2-483c-a000-266d72c8e552 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:30.645133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1141/2274] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:19:30.649706 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d591031-15b7-4ffc-abf5-000c66a3666f admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:30.659894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d591031-15b7-4ffc-abf5-000c66a3666f admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:30.664650 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33fdb3f-86ac-4993-877c-f0e1c73f6337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:30.665323 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33fdb3f-86ac-4993-877c-f0e1c73f6337 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:30.665610 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33fdb3f-86ac-4993-877c-f0e1c73f6337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:30.665890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33fdb3f-86ac-4993-877c-f0e1c73f6337 admin admin] PATH_INFO: `/v3/projects/bc486403073d41609f7b00f732c905ec/users/787e37c0cb2e4e37b67145241eb257fb/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:30.667811 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33fdb3f-86ac-4993-877c-f0e1c73f6337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc486403073d41609f7b00f732c905ec, user_id=787e37c0cb2e4e37b67145241eb257fb, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:30.670436 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33fdb3f-86ac-4993-877c-f0e1c73f6337 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:30.681412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1134/2275] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:19:30 2022] PUT /identity/v3/projects/bc486403073d41609f7b00f732c905ec/users/787e37c0cb2e4e37b67145241eb257fb/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:19:30.691133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ddf0d-4637-4bc1-b852-5df4e171f45e None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:30.691425 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ddf0d-4637-4bc1-b852-5df4e171f45e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:30.691703 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ddf0d-4637-4bc1-b852-5df4e171f45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:30.720329 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-a37ddf0d-4637-4bc1-b852-5df4e171f45e None None] MFA Rules not processed for user `787e37c0cb2e4e37b67145241eb257fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:19:30.721565 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a37ddf0d-4637-4bc1-b852-5df4e171f45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:30.740117 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:19:30.791492 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1142/2276] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:19:30 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:19:30.803727 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33fdb3f-86ac-4993-877c-f0e1c73f6337 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:30.806317 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f33fdb3f-86ac-4993-877c-f0e1c73f6337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:30.863748 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33fdb3f-86ac-4993-877c-f0e1c73f6337 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:30.867407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a057c13-6441-4ded-a573-aa0a74fb7d5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:30.868669 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a057c13-6441-4ded-a573-aa0a74fb7d5b service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:30.869091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a057c13-6441-4ded-a573-aa0a74fb7d5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:30.869367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a057c13-6441-4ded-a573-aa0a74fb7d5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:30.878283 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a057c13-6441-4ded-a573-aa0a74fb7d5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:30.879700 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a057c13-6441-4ded-a573-aa0a74fb7d5b service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:30.889294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1135/2277] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:19:30 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:19:30.941336 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ddf0d-4637-4bc1-b852-5df4e171f45e None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:30.942790 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a37ddf0d-4637-4bc1-b852-5df4e171f45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:30.954670 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ddf0d-4637-4bc1-b852-5df4e171f45e None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:30.958293 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff683d74-8b41-4436-882a-0534987f77fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:30.958886 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff683d74-8b41-4436-882a-0534987f77fb service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:30.959169 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff683d74-8b41-4436-882a-0534987f77fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:30.959446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff683d74-8b41-4436-882a-0534987f77fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:31.015709 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff683d74-8b41-4436-882a-0534987f77fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:31.017181 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff683d74-8b41-4436-882a-0534987f77fb service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:31.026323 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1143/2278] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:19:30 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:19:40.644788 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a057c13-6441-4ded-a573-aa0a74fb7d5b service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:40.706211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a057c13-6441-4ded-a573-aa0a74fb7d5b service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:40.710880 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f00874-fa84-42f1-9ed6-21ffb5995956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:40.711538 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f00874-fa84-42f1-9ed6-21ffb5995956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:40.711822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f00874-fa84-42f1-9ed6-21ffb5995956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:40.712101 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f00874-fa84-42f1-9ed6-21ffb5995956 admin admin] PATH_INFO: `/v3/users/aafd282958b14ae1800408bc8621e870` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:40.713253 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f00874-fa84-42f1-9ed6-21ffb5995956 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aafd282958b14ae1800408bc8621e870)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:40.715184 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f00874-fa84-42f1-9ed6-21ffb5995956 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:40.760707 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-31f00874-fa84-42f1-9ed6-21ffb5995956 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aafd282958b14ae1800408bc8621e870'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:40.767495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-31f00874-fa84-42f1-9ed6-21ffb5995956 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aafd282958b14ae1800408bc8621e870'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:40.774420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:19:40.786970 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-31f00874-fa84-42f1-9ed6-21ffb5995956 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aafd282958b14ae1800408bc8621e870'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:40.787677 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-31f00874-fa84-42f1-9ed6-21ffb5995956 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aafd282958b14ae1800408bc8621e870'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:40.788291 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-31f00874-fa84-42f1-9ed6-21ffb5995956 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aafd282958b14ae1800408bc8621e870'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:40.802515 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1136/2279] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:19:40 2022] DELETE /identity/v3/users/aafd282958b14ae1800408bc8621e870 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:19:40.807195 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8dc24-2fa7-48b6-85d7-934663ea7833 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:40.807822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8dc24-2fa7-48b6-85d7-934663ea7833 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:40.807822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8dc24-2fa7-48b6-85d7-934663ea7833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:40.834032 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-c6e8dc24-2fa7-48b6-85d7-934663ea7833 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:19:40.835237 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c6e8dc24-2fa7-48b6-85d7-934663ea7833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:40.893867 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1144/2280] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:19:40.904159 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f00874-fa84-42f1-9ed6-21ffb5995956 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:40.905609 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-31f00874-fa84-42f1-9ed6-21ffb5995956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:40.964117 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f00874-fa84-42f1-9ed6-21ffb5995956 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:40.968591 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f4a6d-aa19-4613-a977-0ff5fe3a3d39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:40.969259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f4a6d-aa19-4613-a977-0ff5fe3a3d39 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:40.969550 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f4a6d-aa19-4613-a977-0ff5fe3a3d39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:40.969827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f4a6d-aa19-4613-a977-0ff5fe3a3d39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:40.979640 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f4a6d-aa19-4613-a977-0ff5fe3a3d39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:40.982007 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f4a6d-aa19-4613-a977-0ff5fe3a3d39 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:40.992339 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1137/2281] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:19:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:19:41.265873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8dc24-2fa7-48b6-85d7-934663ea7833 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:41.267461 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c6e8dc24-2fa7-48b6-85d7-934663ea7833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:41.281592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8dc24-2fa7-48b6-85d7-934663ea7833 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:41.286929 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a0759-75f9-4695-875a-28f350d07c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:41.287812 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a0759-75f9-4695-875a-28f350d07c74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:41.288158 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a0759-75f9-4695-875a-28f350d07c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:41.288365 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a0759-75f9-4695-875a-28f350d07c74 admin admin] PATH_INFO: `/v3/projects/342b9acebd1246d2bf62664e062fa4d7` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:41.289574 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a0759-75f9-4695-875a-28f350d07c74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=342b9acebd1246d2bf62664e062fa4d7)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:41.291494 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a0759-75f9-4695-875a-28f350d07c74 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:41.339401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-874a0759-75f9-4695-875a-28f350d07c74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '342b9acebd1246d2bf62664e062fa4d7', 'request_id': 'req-874a0759-75f9-4695-875a-28f350d07c74'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:41.341300 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:19:41.347555 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-874a0759-75f9-4695-875a-28f350d07c74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '342b9acebd1246d2bf62664e062fa4d7', 'request_id': 'req-874a0759-75f9-4695-875a-28f350d07c74'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:41.353384 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-874a0759-75f9-4695-875a-28f350d07c74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '342b9acebd1246d2bf62664e062fa4d7', 'request_id': 'req-874a0759-75f9-4695-875a-28f350d07c74'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:41.365091 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-874a0759-75f9-4695-875a-28f350d07c74 admin admin] The token cache is being invalidate because project 342b9acebd1246d2bf62664e062fa4d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:19:41.365378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-874a0759-75f9-4695-875a-28f350d07c74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:41.366007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-874a0759-75f9-4695-875a-28f350d07c74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:41.367473 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1145/2282] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:19:41 2022] DELETE /identity/v3/projects/342b9acebd1246d2bf62664e062fa4d7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:19:41.372085 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f4a6d-aa19-4613-a977-0ff5fe3a3d39 service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:41.381571 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f4a6d-aa19-4613-a977-0ff5fe3a3d39 service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:41.436342 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6355ae2-1f6a-46be-9304-7ec83ad3356e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:41.436989 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6355ae2-1f6a-46be-9304-7ec83ad3356e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:41.437271 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6355ae2-1f6a-46be-9304-7ec83ad3356e admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:41.437547 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6355ae2-1f6a-46be-9304-7ec83ad3356e admin admin] PATH_INFO: `/v3/users/787e37c0cb2e4e37b67145241eb257fb` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:41.438643 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6355ae2-1f6a-46be-9304-7ec83ad3356e admin admin] RBAC: Authorizing `identity:delete_user(user_id=787e37c0cb2e4e37b67145241eb257fb)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:41.440562 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6355ae2-1f6a-46be-9304-7ec83ad3356e admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:41.485790 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f6355ae2-1f6a-46be-9304-7ec83ad3356e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '787e37c0cb2e4e37b67145241eb257fb'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:41.492941 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f6355ae2-1f6a-46be-9304-7ec83ad3356e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '787e37c0cb2e4e37b67145241eb257fb'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:41.505727 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f6355ae2-1f6a-46be-9304-7ec83ad3356e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '787e37c0cb2e4e37b67145241eb257fb'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:41.506402 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f6355ae2-1f6a-46be-9304-7ec83ad3356e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '787e37c0cb2e4e37b67145241eb257fb'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:41.507018 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f6355ae2-1f6a-46be-9304-7ec83ad3356e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '787e37c0cb2e4e37b67145241eb257fb'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:41.510540 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:19:41.520980 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1138/2283] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:19:41 2022] DELETE /identity/v3/users/787e37c0cb2e4e37b67145241eb257fb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:19:41.797242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a0759-75f9-4695-875a-28f350d07c74 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:41.798913 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-874a0759-75f9-4695-875a-28f350d07c74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:41.815904 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a0759-75f9-4695-875a-28f350d07c74 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:41.879851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9682a-0536-480a-a9b8-5dec9a7a2f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:41.880417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9682a-0536-480a-a9b8-5dec9a7a2f7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:41.880736 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9682a-0536-480a-a9b8-5dec9a7a2f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:41.881038 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9682a-0536-480a-a9b8-5dec9a7a2f7d admin admin] PATH_INFO: `/v3/projects/bc486403073d41609f7b00f732c905ec` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:41.882221 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9682a-0536-480a-a9b8-5dec9a7a2f7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc486403073d41609f7b00f732c905ec)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:41.884255 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9682a-0536-480a-a9b8-5dec9a7a2f7d admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:41.932101 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-63f9682a-0536-480a-a9b8-5dec9a7a2f7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc486403073d41609f7b00f732c905ec', 'request_id': 'req-63f9682a-0536-480a-a9b8-5dec9a7a2f7d'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:41.940128 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-63f9682a-0536-480a-a9b8-5dec9a7a2f7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc486403073d41609f7b00f732c905ec', 'request_id': 'req-63f9682a-0536-480a-a9b8-5dec9a7a2f7d'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:41.945774 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-63f9682a-0536-480a-a9b8-5dec9a7a2f7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc486403073d41609f7b00f732c905ec', 'request_id': 'req-63f9682a-0536-480a-a9b8-5dec9a7a2f7d'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:41.957112 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-63f9682a-0536-480a-a9b8-5dec9a7a2f7d admin admin] The token cache is being invalidate because project bc486403073d41609f7b00f732c905ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:19:41.957402 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-63f9682a-0536-480a-a9b8-5dec9a7a2f7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:41.958031 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-63f9682a-0536-480a-a9b8-5dec9a7a2f7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:19:41.959517 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1146/2284] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:19:41 2022] DELETE /identity/v3/projects/bc486403073d41609f7b00f732c905ec => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:19:41.964329 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebb535-ec4d-49a1-912b-b4fb97123382 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:41.964928 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebb535-ec4d-49a1-912b-b4fb97123382 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:41.965171 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebb535-ec4d-49a1-912b-b4fb97123382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:41.984240 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-d2ebb535-ec4d-49a1-912b-b4fb97123382 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:19:41.985481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d2ebb535-ec4d-49a1-912b-b4fb97123382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:42.044266 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1139/2285] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:19:42.052048 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13b8e4-3d3b-40c5-ba4c-1c944972a4c5 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:42.052344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13b8e4-3d3b-40c5-ba4c-1c944972a4c5 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:42.052677 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13b8e4-3d3b-40c5-ba4c-1c944972a4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:42.074028 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-ea13b8e4-3d3b-40c5-ba4c-1c944972a4c5 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:19:42.075241 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ea13b8e4-3d3b-40c5-ba4c-1c944972a4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:42.083133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1147/2286] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:19:42.087322 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ebb535-ec4d-49a1-912b-b4fb97123382 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:42.097219 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ebb535-ec4d-49a1-912b-b4fb97123382 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:42.101794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4a2be-ce00-4d99-9d58-31f465c5d931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:42.102412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4a2be-ce00-4d99-9d58-31f465c5d931 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:42.102693 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4a2be-ce00-4d99-9d58-31f465c5d931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:42.102969 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4a2be-ce00-4d99-9d58-31f465c5d931 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:42.103618 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4a2be-ce00-4d99-9d58-31f465c5d931 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:19:42.103924 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4a2be-ce00-4d99-9d58-31f465c5d931 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:42.106104 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4a2be-ce00-4d99-9d58-31f465c5d931 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:42.118398 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1140/2287] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:19:42.123293 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13b8e4-3d3b-40c5-ba4c-1c944972a4c5 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:42.132476 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13b8e4-3d3b-40c5-ba4c-1c944972a4c5 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:42.137273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434bbeda-4154-4dd2-ac42-699ce4d658dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:42.137870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434bbeda-4154-4dd2-ac42-699ce4d658dd admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:42.138180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434bbeda-4154-4dd2-ac42-699ce4d658dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:42.138450 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434bbeda-4154-4dd2-ac42-699ce4d658dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:42.139087 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434bbeda-4154-4dd2-ac42-699ce4d658dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:42.141007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434bbeda-4154-4dd2-ac42-699ce4d658dd admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:42.169488 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1148/2288] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:19:42 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:19:42.173669 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4a2be-ce00-4d99-9d58-31f465c5d931 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:42.183458 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4a2be-ce00-4d99-9d58-31f465c5d931 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:42.238906 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c49a0-9eb3-417e-9675-85eba0a89975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:42.239589 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c49a0-9eb3-417e-9675-85eba0a89975 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:42.239873 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c49a0-9eb3-417e-9675-85eba0a89975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:42.240153 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c49a0-9eb3-417e-9675-85eba0a89975 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:42.240823 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c49a0-9eb3-417e-9675-85eba0a89975 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:42.242711 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87c49a0-9eb3-417e-9675-85eba0a89975 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:42.264764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1141/2289] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:19:42 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:19:42.268880 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434bbeda-4154-4dd2-ac42-699ce4d658dd admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:42.279643 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434bbeda-4154-4dd2-ac42-699ce4d658dd admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:42.284687 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8439e95-3c93-4771-90e0-8fec1ad87efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:42.285294 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8439e95-3c93-4771-90e0-8fec1ad87efe admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:42.285579 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8439e95-3c93-4771-90e0-8fec1ad87efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:42.285858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8439e95-3c93-4771-90e0-8fec1ad87efe admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:42.286500 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8439e95-3c93-4771-90e0-8fec1ad87efe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:42.288093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8439e95-3c93-4771-90e0-8fec1ad87efe admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:42.302738 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1149/2290] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:19:42.303874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c49a0-9eb3-417e-9675-85eba0a89975 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:42.314917 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c49a0-9eb3-417e-9675-85eba0a89975 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:42.319893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c6955-f07a-4203-865a-dc9ec6c941d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:42.320669 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c6955-f07a-4203-865a-dc9ec6c941d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:42.320986 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c6955-f07a-4203-865a-dc9ec6c941d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:42.321268 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c6955-f07a-4203-865a-dc9ec6c941d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:42.321912 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c6955-f07a-4203-865a-dc9ec6c941d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:42.323526 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c6955-f07a-4203-865a-dc9ec6c941d4 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:42.338584 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1142/2291] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:19:42.339574 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8439e95-3c93-4771-90e0-8fec1ad87efe admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:42.349908 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8439e95-3c93-4771-90e0-8fec1ad87efe admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:42.355251 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc733c-a705-40b4-b322-f6fffb6dc65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:42.355903 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc733c-a705-40b4-b322-f6fffb6dc65c admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:42.356794 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc733c-a705-40b4-b322-f6fffb6dc65c admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:42.357246 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc733c-a705-40b4-b322-f6fffb6dc65c admin admin] PATH_INFO: `/v3/projects/e4d4de8729614585818cdb9d325c54c2/users/18ee112cfcf643578474a181979bea83/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:42.374227 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc733c-a705-40b4-b322-f6fffb6dc65c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4d4de8729614585818cdb9d325c54c2, user_id=18ee112cfcf643578474a181979bea83, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:42.376995 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc733c-a705-40b4-b322-f6fffb6dc65c admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:42.389889 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1150/2292] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:19:42 2022] PUT /identity/v3/projects/e4d4de8729614585818cdb9d325c54c2/users/18ee112cfcf643578474a181979bea83/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:19:42.408275 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb27e8-6a3a-4526-af77-b81cd18aed32 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:42.408697 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb27e8-6a3a-4526-af77-b81cd18aed32 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:42.408991 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb27e8-6a3a-4526-af77-b81cd18aed32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:42.438859 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-9ccb27e8-6a3a-4526-af77-b81cd18aed32 None None] MFA Rules not processed for user `18ee112cfcf643578474a181979bea83`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:19:42.440315 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9ccb27e8-6a3a-4526-af77-b81cd18aed32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:42.490326 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:19:42.509573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1143/2293] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:19:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:19:42.521191 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc733c-a705-40b4-b322-f6fffb6dc65c admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:19:42.522964 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a7cc733c-a705-40b4-b322-f6fffb6dc65c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:19:42.530702 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:19:42.591717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc733c-a705-40b4-b322-f6fffb6dc65c admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:19:42.595958 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6497b-6292-4ab5-a59a-499d992d4c35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:19:42.596597 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6497b-6292-4ab5-a59a-499d992d4c35 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:19:42.596883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6497b-6292-4ab5-a59a-499d992d4c35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:19:42.597159 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6497b-6292-4ab5-a59a-499d992d4c35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:19:42.606119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c6497b-6292-4ab5-a59a-499d992d4c35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:19:42.608396 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c6497b-6292-4ab5-a59a-499d992d4c35 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:19:42.617857 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1151/2294] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:19:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:19:52.506252 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=4 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:19:52.613739 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=24 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:02.053460 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb27e8-6a3a-4526-af77-b81cd18aed32 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:02.054891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9ccb27e8-6a3a-4526-af77-b81cd18aed32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:02.120411 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb27e8-6a3a-4526-af77-b81cd18aed32 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:02.125544 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13ab93-28af-4d61-9f3c-9dbca0ab7988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:02.126170 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13ab93-28af-4d61-9f3c-9dbca0ab7988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:02.126462 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13ab93-28af-4d61-9f3c-9dbca0ab7988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:02.126743 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13ab93-28af-4d61-9f3c-9dbca0ab7988 admin admin] PATH_INFO: `/v3/users/95c411770572495d95aaca929c3e33cc` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:02.127871 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13ab93-28af-4d61-9f3c-9dbca0ab7988 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95c411770572495d95aaca929c3e33cc)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:02.129800 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13ab93-28af-4d61-9f3c-9dbca0ab7988 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:02.175888 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-5e13ab93-28af-4d61-9f3c-9dbca0ab7988 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95c411770572495d95aaca929c3e33cc'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:02.182851 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-5e13ab93-28af-4d61-9f3c-9dbca0ab7988 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95c411770572495d95aaca929c3e33cc'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:02.195050 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-5e13ab93-28af-4d61-9f3c-9dbca0ab7988 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95c411770572495d95aaca929c3e33cc'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:02.195675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-5e13ab93-28af-4d61-9f3c-9dbca0ab7988 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95c411770572495d95aaca929c3e33cc'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:02.196275 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-5e13ab93-28af-4d61-9f3c-9dbca0ab7988 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95c411770572495d95aaca929c3e33cc'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:02.209384 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1144/2295] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:20:02 2022] DELETE /identity/v3/users/95c411770572495d95aaca929c3e33cc => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:02.213993 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f984bbd-e02c-4e37-a0dd-9a84bf6273f3 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:02.214488 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f984bbd-e02c-4e37-a0dd-9a84bf6273f3 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:02.214620 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f984bbd-e02c-4e37-a0dd-9a84bf6273f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:02.233593 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-9f984bbd-e02c-4e37-a0dd-9a84bf6273f3 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:02.234785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9f984bbd-e02c-4e37-a0dd-9a84bf6273f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:02.298014 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1152/2296] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:02.307889 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13ab93-28af-4d61-9f3c-9dbca0ab7988 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:02.309313 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5e13ab93-28af-4d61-9f3c-9dbca0ab7988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:02.367112 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13ab93-28af-4d61-9f3c-9dbca0ab7988 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:02.370763 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee35af1-0108-40a4-b329-4489be375666 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:02.371355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee35af1-0108-40a4-b329-4489be375666 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:02.371637 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee35af1-0108-40a4-b329-4489be375666 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:02.371910 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee35af1-0108-40a4-b329-4489be375666 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:02.381911 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee35af1-0108-40a4-b329-4489be375666 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:02.383312 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee35af1-0108-40a4-b329-4489be375666 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:02.392566 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1145/2297] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:20:02.654957 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f984bbd-e02c-4e37-a0dd-9a84bf6273f3 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:02.656448 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9f984bbd-e02c-4e37-a0dd-9a84bf6273f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:02.669181 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f984bbd-e02c-4e37-a0dd-9a84bf6273f3 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:02.674717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f8d25-169a-4439-b5d0-1c95c09f071b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:02.675352 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f8d25-169a-4439-b5d0-1c95c09f071b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:02.675638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f8d25-169a-4439-b5d0-1c95c09f071b admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:02.675915 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f8d25-169a-4439-b5d0-1c95c09f071b admin admin] PATH_INFO: `/v3/projects/c48287b44b52498da1875ce87b63d3bd` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:02.677070 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f8d25-169a-4439-b5d0-1c95c09f071b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c48287b44b52498da1875ce87b63d3bd)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:02.678970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f8d25-169a-4439-b5d0-1c95c09f071b admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:02.725769 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-229f8d25-169a-4439-b5d0-1c95c09f071b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c48287b44b52498da1875ce87b63d3bd', 'request_id': 'req-229f8d25-169a-4439-b5d0-1c95c09f071b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:02.733178 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-229f8d25-169a-4439-b5d0-1c95c09f071b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c48287b44b52498da1875ce87b63d3bd', 'request_id': 'req-229f8d25-169a-4439-b5d0-1c95c09f071b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:02.739545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-229f8d25-169a-4439-b5d0-1c95c09f071b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c48287b44b52498da1875ce87b63d3bd', 'request_id': 'req-229f8d25-169a-4439-b5d0-1c95c09f071b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:02.750883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-229f8d25-169a-4439-b5d0-1c95c09f071b admin admin] The token cache is being invalidate because project c48287b44b52498da1875ce87b63d3bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:20:02.751176 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-229f8d25-169a-4439-b5d0-1c95c09f071b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:02.751804 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-229f8d25-169a-4439-b5d0-1c95c09f071b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:02.753302 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1153/2298] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:20:02 2022] DELETE /identity/v3/projects/c48287b44b52498da1875ce87b63d3bd => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:02.757288 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee35af1-0108-40a4-b329-4489be375666 service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:02.767938 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee35af1-0108-40a4-b329-4489be375666 service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:02.786204 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:02.826741 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ad7df-8604-4d45-ad44-8affafe73752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:02.827371 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ad7df-8604-4d45-ad44-8affafe73752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:02.827654 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ad7df-8604-4d45-ad44-8affafe73752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:02.827931 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ad7df-8604-4d45-ad44-8affafe73752 admin admin] PATH_INFO: `/v3/users/3a34d4b0b6c54db8b5e9fb1aca42966e` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:02.829087 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ad7df-8604-4d45-ad44-8affafe73752 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a34d4b0b6c54db8b5e9fb1aca42966e)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:02.830986 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ad7df-8604-4d45-ad44-8affafe73752 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:02.876688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-082ad7df-8604-4d45-ad44-8affafe73752 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a34d4b0b6c54db8b5e9fb1aca42966e'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:02.883439 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-082ad7df-8604-4d45-ad44-8affafe73752 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a34d4b0b6c54db8b5e9fb1aca42966e'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:02.896547 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-082ad7df-8604-4d45-ad44-8affafe73752 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a34d4b0b6c54db8b5e9fb1aca42966e'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:02.897225 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-082ad7df-8604-4d45-ad44-8affafe73752 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a34d4b0b6c54db8b5e9fb1aca42966e'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:02.897831 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-082ad7df-8604-4d45-ad44-8affafe73752 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a34d4b0b6c54db8b5e9fb1aca42966e'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:02.911263 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1146/2299] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:20:02 2022] DELETE /identity/v3/users/3a34d4b0b6c54db8b5e9fb1aca42966e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:03.177925 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f8d25-169a-4439-b5d0-1c95c09f071b admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:03.179429 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-229f8d25-169a-4439-b5d0-1c95c09f071b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:03.193166 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f8d25-169a-4439-b5d0-1c95c09f071b admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:03.249978 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c97e4c-d01c-45be-8d97-4378f0d7447b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:03.250652 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c97e4c-d01c-45be-8d97-4378f0d7447b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:03.250862 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c97e4c-d01c-45be-8d97-4378f0d7447b admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:03.251135 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c97e4c-d01c-45be-8d97-4378f0d7447b admin admin] PATH_INFO: `/v3/projects/107446a8e1d543eaa50ac1b877ce4c84` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:03.252232 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c97e4c-d01c-45be-8d97-4378f0d7447b admin admin] RBAC: Authorizing `identity:delete_project(project_id=107446a8e1d543eaa50ac1b877ce4c84)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:03.255019 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c97e4c-d01c-45be-8d97-4378f0d7447b admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:03.285137 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:03.302262 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-44c97e4c-d01c-45be-8d97-4378f0d7447b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '107446a8e1d543eaa50ac1b877ce4c84', 'request_id': 'req-44c97e4c-d01c-45be-8d97-4378f0d7447b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:03.309314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-44c97e4c-d01c-45be-8d97-4378f0d7447b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '107446a8e1d543eaa50ac1b877ce4c84', 'request_id': 'req-44c97e4c-d01c-45be-8d97-4378f0d7447b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:03.314976 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-44c97e4c-d01c-45be-8d97-4378f0d7447b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '107446a8e1d543eaa50ac1b877ce4c84', 'request_id': 'req-44c97e4c-d01c-45be-8d97-4378f0d7447b'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:03.326329 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-44c97e4c-d01c-45be-8d97-4378f0d7447b admin admin] The token cache is being invalidate because project 107446a8e1d543eaa50ac1b877ce4c84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:20:03.326329 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-44c97e4c-d01c-45be-8d97-4378f0d7447b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:03.326975 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-44c97e4c-d01c-45be-8d97-4378f0d7447b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:03.328429 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1154/2300] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:20:03 2022] DELETE /identity/v3/projects/107446a8e1d543eaa50ac1b877ce4c84 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:03.334019 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdd7b6-e7fd-4ec4-8bba-a1b8a62e613a None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:03.334527 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdd7b6-e7fd-4ec4-8bba-a1b8a62e613a None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:03.334631 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdd7b6-e7fd-4ec4-8bba-a1b8a62e613a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:03.354850 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1cbdd7b6-e7fd-4ec4-8bba-a1b8a62e613a None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:03.356061 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1cbdd7b6-e7fd-4ec4-8bba-a1b8a62e613a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:03.414943 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1147/2301] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:03.423026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0965c-90fe-4f2c-90c9-2a8a9b3a762f None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:03.423316 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0965c-90fe-4f2c-90c9-2a8a9b3a762f None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:03.423597 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0965c-90fe-4f2c-90c9-2a8a9b3a762f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:03.443637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-e4d0965c-90fe-4f2c-90c9-2a8a9b3a762f None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:03.444893 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-e4d0965c-90fe-4f2c-90c9-2a8a9b3a762f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:03.453437 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1155/2302] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:03.457530 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdd7b6-e7fd-4ec4-8bba-a1b8a62e613a None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:03.467250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdd7b6-e7fd-4ec4-8bba-a1b8a62e613a None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:03.472039 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291963c0-31b7-487a-b38a-1d44c197619a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:03.472593 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291963c0-31b7-487a-b38a-1d44c197619a admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:03.472880 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291963c0-31b7-487a-b38a-1d44c197619a admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:03.473166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291963c0-31b7-487a-b38a-1d44c197619a admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:03.473808 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291963c0-31b7-487a-b38a-1d44c197619a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:20:03.474116 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291963c0-31b7-487a-b38a-1d44c197619a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:03.475697 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291963c0-31b7-487a-b38a-1d44c197619a admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:03.487395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1148/2303] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:20:03.491946 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0965c-90fe-4f2c-90c9-2a8a9b3a762f None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:03.501685 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0965c-90fe-4f2c-90c9-2a8a9b3a762f None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:03.506306 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f470f0d-263d-4626-b0ee-99e8568eb1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:03.506896 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f470f0d-263d-4626-b0ee-99e8568eb1b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:03.507182 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f470f0d-263d-4626-b0ee-99e8568eb1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:03.507459 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f470f0d-263d-4626-b0ee-99e8568eb1b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:03.508083 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f470f0d-263d-4626-b0ee-99e8568eb1b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:03.510082 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f470f0d-263d-4626-b0ee-99e8568eb1b8 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:03.536838 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1156/2304] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:20:03 2022] POST /identity/v3/projects => generated 399 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:20:03.540875 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291963c0-31b7-487a-b38a-1d44c197619a admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:03.551054 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291963c0-31b7-487a-b38a-1d44c197619a admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:03.578153 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:03.616353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31332626-74b1-47f1-b2c2-888fdd63e6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:03.617138 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31332626-74b1-47f1-b2c2-888fdd63e6c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:03.617297 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31332626-74b1-47f1-b2c2-888fdd63e6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:03.617576 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31332626-74b1-47f1-b2c2-888fdd63e6c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:03.618217 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31332626-74b1-47f1-b2c2-888fdd63e6c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:03.620126 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31332626-74b1-47f1-b2c2-888fdd63e6c8 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:03.642945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1149/2305] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:20:03 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:20:03.647064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f470f0d-263d-4626-b0ee-99e8568eb1b8 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:03.657392 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f470f0d-263d-4626-b0ee-99e8568eb1b8 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:03.662090 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64536a7-317d-4df3-883d-4f7a5fa62fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:03.662681 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64536a7-317d-4df3-883d-4f7a5fa62fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:03.662963 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64536a7-317d-4df3-883d-4f7a5fa62fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:03.663239 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64536a7-317d-4df3-883d-4f7a5fa62fb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:03.663878 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64536a7-317d-4df3-883d-4f7a5fa62fb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:03.665497 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64536a7-317d-4df3-883d-4f7a5fa62fb7 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:03.675458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1157/2306] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:20:03.679441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31332626-74b1-47f1-b2c2-888fdd63e6c8 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:03.690752 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31332626-74b1-47f1-b2c2-888fdd63e6c8 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:03.696088 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55ea44-5a8e-4979-80f3-e22d512ae63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:03.696732 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55ea44-5a8e-4979-80f3-e22d512ae63c admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:03.697014 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55ea44-5a8e-4979-80f3-e22d512ae63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:03.697288 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55ea44-5a8e-4979-80f3-e22d512ae63c admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:03.697932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e55ea44-5a8e-4979-80f3-e22d512ae63c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:03.699535 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e55ea44-5a8e-4979-80f3-e22d512ae63c admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:03.710579 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1150/2307] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:20:03.714692 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64536a7-317d-4df3-883d-4f7a5fa62fb7 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:03.724681 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64536a7-317d-4df3-883d-4f7a5fa62fb7 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:03.729250 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b173b8d-d060-4e3b-8e44-60e2af6f6932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:03.729898 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b173b8d-d060-4e3b-8e44-60e2af6f6932 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:03.730182 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b173b8d-d060-4e3b-8e44-60e2af6f6932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:03.730462 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b173b8d-d060-4e3b-8e44-60e2af6f6932 admin admin] PATH_INFO: `/v3/projects/59c03b73d33b4a1cab20042f3a9729aa/users/f4b90d76ecd143738e7094908960d5dd/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:03.745323 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b173b8d-d060-4e3b-8e44-60e2af6f6932 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59c03b73d33b4a1cab20042f3a9729aa, user_id=f4b90d76ecd143738e7094908960d5dd, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:03.748653 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b173b8d-d060-4e3b-8e44-60e2af6f6932 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:03.761225 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1158/2308] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:20:03 2022] PUT /identity/v3/projects/59c03b73d33b4a1cab20042f3a9729aa/users/f4b90d76ecd143738e7094908960d5dd/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:03.773284 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef47f68-fb7b-4f8b-a601-68f9002860dc None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:03.773570 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef47f68-fb7b-4f8b-a601-68f9002860dc None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:03.773844 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef47f68-fb7b-4f8b-a601-68f9002860dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:03.801526 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-0ef47f68-fb7b-4f8b-a601-68f9002860dc None None] MFA Rules not processed for user `f4b90d76ecd143738e7094908960d5dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:03.802648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-0ef47f68-fb7b-4f8b-a601-68f9002860dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:03.867564 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1151/2309] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:03 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:03.872012 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b173b8d-d060-4e3b-8e44-60e2af6f6932 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:03.934120 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b173b8d-d060-4e3b-8e44-60e2af6f6932 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:03.938792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f658c9-982f-45c9-a6a6-67aae0f4fc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:03.939391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f658c9-982f-45c9-a6a6-67aae0f4fc43 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:03.939673 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f658c9-982f-45c9-a6a6-67aae0f4fc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:03.939949 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f658c9-982f-45c9-a6a6-67aae0f4fc43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:03.940622 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f658c9-982f-45c9-a6a6-67aae0f4fc43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:03.942548 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f658c9-982f-45c9-a6a6-67aae0f4fc43 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:03.969297 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1159/2310] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:20:03 2022] POST /identity/v3/projects => generated 401 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:20:03.973389 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef47f68-fb7b-4f8b-a601-68f9002860dc None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:03.983935 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef47f68-fb7b-4f8b-a601-68f9002860dc None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:04.038875 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f897c433-7a97-467f-ad14-2ddd5bcfc92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:04.039500 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897c433-7a97-467f-ad14-2ddd5bcfc92a admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:04.039781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897c433-7a97-467f-ad14-2ddd5bcfc92a admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:04.040054 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897c433-7a97-467f-ad14-2ddd5bcfc92a admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:04.040710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f897c433-7a97-467f-ad14-2ddd5bcfc92a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:04.042602 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f897c433-7a97-467f-ad14-2ddd5bcfc92a admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:04.063476 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1152/2311] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:20:03 2022] POST /identity/v3/users => generated 307 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:20:04.067453 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f658c9-982f-45c9-a6a6-67aae0f4fc43 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:04.078224 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f658c9-982f-45c9-a6a6-67aae0f4fc43 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:04.082806 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978992d9-9c81-4e44-a8d7-37d6fa28659b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:04.083399 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978992d9-9c81-4e44-a8d7-37d6fa28659b admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:04.083683 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978992d9-9c81-4e44-a8d7-37d6fa28659b admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:04.083960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978992d9-9c81-4e44-a8d7-37d6fa28659b admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:04.084635 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978992d9-9c81-4e44-a8d7-37d6fa28659b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:04.086235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978992d9-9c81-4e44-a8d7-37d6fa28659b admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:04.097607 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1160/2312] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:20:04.101752 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f897c433-7a97-467f-ad14-2ddd5bcfc92a admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:04.112606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f897c433-7a97-467f-ad14-2ddd5bcfc92a admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:04.117620 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b2e4c-65d2-487f-9a52-75d506a53fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:04.118270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b2e4c-65d2-487f-9a52-75d506a53fd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:04.118555 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b2e4c-65d2-487f-9a52-75d506a53fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:04.118834 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b2e4c-65d2-487f-9a52-75d506a53fd3 admin admin] PATH_INFO: `/v3/projects/d23f3f29f8834a59b885c8e8359fefb8/users/e54e365ba71a4d0b8e99288c34c64180/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:04.133865 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b2e4c-65d2-487f-9a52-75d506a53fd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d23f3f29f8834a59b885c8e8359fefb8, user_id=e54e365ba71a4d0b8e99288c34c64180, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:04.136481 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b2e4c-65d2-487f-9a52-75d506a53fd3 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:04.149603 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1153/2313] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:20:04 2022] PUT /identity/v3/projects/d23f3f29f8834a59b885c8e8359fefb8/users/e54e365ba71a4d0b8e99288c34c64180/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:04.153824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978992d9-9c81-4e44-a8d7-37d6fa28659b admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:04.163763 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978992d9-9c81-4e44-a8d7-37d6fa28659b admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:04.183481 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:04.222387 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead67d89-4be1-43a1-86a1-e0f933a49d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:04.222995 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead67d89-4be1-43a1-86a1-e0f933a49d62 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:04.223275 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead67d89-4be1-43a1-86a1-e0f933a49d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:04.223546 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead67d89-4be1-43a1-86a1-e0f933a49d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:04.224176 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead67d89-4be1-43a1-86a1-e0f933a49d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:04.225783 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead67d89-4be1-43a1-86a1-e0f933a49d62 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:04.235286 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1161/2314] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:20:04.239627 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b2e4c-65d2-487f-9a52-75d506a53fd3 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:04.245004 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:04.252582 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b2e4c-65d2-487f-9a52-75d506a53fd3 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:04.257734 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc37075-0110-40a1-ba0a-1428cc59bb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:04.258385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc37075-0110-40a1-ba0a-1428cc59bb04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:04.258670 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc37075-0110-40a1-ba0a-1428cc59bb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:04.258973 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc37075-0110-40a1-ba0a-1428cc59bb04 admin admin] PATH_INFO: `/v3/projects/d23f3f29f8834a59b885c8e8359fefb8/users/e54e365ba71a4d0b8e99288c34c64180/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:04.260979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc37075-0110-40a1-ba0a-1428cc59bb04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d23f3f29f8834a59b885c8e8359fefb8, user_id=e54e365ba71a4d0b8e99288c34c64180, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:04.264307 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc37075-0110-40a1-ba0a-1428cc59bb04 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:04.274473 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1154/2315] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:20:04 2022] PUT /identity/v3/projects/d23f3f29f8834a59b885c8e8359fefb8/users/e54e365ba71a4d0b8e99288c34c64180/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:04.284714 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0561d-2d95-4ff7-84b3-62874a5b468e None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:04.285219 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0561d-2d95-4ff7-84b3-62874a5b468e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:04.285388 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0561d-2d95-4ff7-84b3-62874a5b468e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:04.313088 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-27e0561d-2d95-4ff7-84b3-62874a5b468e None None] MFA Rules not processed for user `e54e365ba71a4d0b8e99288c34c64180`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:04.314328 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-27e0561d-2d95-4ff7-84b3-62874a5b468e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:04.380811 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1162/2316] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:04 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:04.392036 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc37075-0110-40a1-ba0a-1428cc59bb04 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:04.393927 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ddc37075-0110-40a1-ba0a-1428cc59bb04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:04.451453 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc37075-0110-40a1-ba0a-1428cc59bb04 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:04.455357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a75274-8154-42e5-afe6-53d3220ebf8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:04.455988 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a75274-8154-42e5-afe6-53d3220ebf8c service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:04.456270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a75274-8154-42e5-afe6-53d3220ebf8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:04.456575 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a75274-8154-42e5-afe6-53d3220ebf8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:04.465870 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a75274-8154-42e5-afe6-53d3220ebf8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:04.467408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a75274-8154-42e5-afe6-53d3220ebf8c service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:04.476682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1155/2317] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:04 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:20:04.551688 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0561d-2d95-4ff7-84b3-62874a5b468e None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:04.562161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0561d-2d95-4ff7-84b3-62874a5b468e None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:04.565869 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b38c7f-6dbd-40d5-921b-6b663da09039 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:04.566411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b38c7f-6dbd-40d5-921b-6b663da09039 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:04.566694 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b38c7f-6dbd-40d5-921b-6b663da09039 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:04.566996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b38c7f-6dbd-40d5-921b-6b663da09039 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:04.624730 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b38c7f-6dbd-40d5-921b-6b663da09039 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:04.626193 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b38c7f-6dbd-40d5-921b-6b663da09039 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:04.635638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1163/2318] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:04 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:20:09.750106 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a75274-8154-42e5-afe6-53d3220ebf8c service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:09.813378 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a75274-8154-42e5-afe6-53d3220ebf8c service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:09.817950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fdb068-28dd-489e-b00c-f422ab18792e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:09.818574 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fdb068-28dd-489e-b00c-f422ab18792e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:09.818859 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fdb068-28dd-489e-b00c-f422ab18792e admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:09.819141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fdb068-28dd-489e-b00c-f422ab18792e admin admin] PATH_INFO: `/v3/users/f4b90d76ecd143738e7094908960d5dd` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:09.820254 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fdb068-28dd-489e-b00c-f422ab18792e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4b90d76ecd143738e7094908960d5dd)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:09.822187 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fdb068-28dd-489e-b00c-f422ab18792e admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:09.867477 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c1fdb068-28dd-489e-b00c-f422ab18792e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4b90d76ecd143738e7094908960d5dd'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:09.875815 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c1fdb068-28dd-489e-b00c-f422ab18792e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4b90d76ecd143738e7094908960d5dd'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:09.887792 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c1fdb068-28dd-489e-b00c-f422ab18792e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4b90d76ecd143738e7094908960d5dd'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:09.888613 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c1fdb068-28dd-489e-b00c-f422ab18792e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4b90d76ecd143738e7094908960d5dd'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:09.889314 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-c1fdb068-28dd-489e-b00c-f422ab18792e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4b90d76ecd143738e7094908960d5dd'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:09.901808 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1156/2319] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:20:09 2022] DELETE /identity/v3/users/f4b90d76ecd143738e7094908960d5dd => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:09.906588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca0ee9-d899-442e-884b-4199d0706317 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:09.907073 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca0ee9-d899-442e-884b-4199d0706317 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:09.907183 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca0ee9-d899-442e-884b-4199d0706317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:09.926280 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-75ca0ee9-d899-442e-884b-4199d0706317 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:09.927482 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-75ca0ee9-d899-442e-884b-4199d0706317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:09.966636 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=100 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:09.986292 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1164/2320] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:09.997045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fdb068-28dd-489e-b00c-f422ab18792e admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:09.999707 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-c1fdb068-28dd-489e-b00c-f422ab18792e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:10.013182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:10.057278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fdb068-28dd-489e-b00c-f422ab18792e admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:10.061622 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa053a-a0a8-432c-a98f-11aa1b3d56f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:10.062224 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa053a-a0a8-432c-a98f-11aa1b3d56f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:10.062510 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa053a-a0a8-432c-a98f-11aa1b3d56f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:10.062793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa053a-a0a8-432c-a98f-11aa1b3d56f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:10.071677 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa053a-a0a8-432c-a98f-11aa1b3d56f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:10.073136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa053a-a0a8-432c-a98f-11aa1b3d56f9 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:10.082494 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1157/2321] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:20:10.347378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca0ee9-d899-442e-884b-4199d0706317 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:10.348867 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-75ca0ee9-d899-442e-884b-4199d0706317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:10.361499 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca0ee9-d899-442e-884b-4199d0706317 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:10.367255 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace99169-3b29-46d7-a95b-58b8233b172d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:10.368171 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace99169-3b29-46d7-a95b-58b8233b172d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:10.368466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace99169-3b29-46d7-a95b-58b8233b172d admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:10.368776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace99169-3b29-46d7-a95b-58b8233b172d admin admin] PATH_INFO: `/v3/projects/59c03b73d33b4a1cab20042f3a9729aa` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:10.369920 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace99169-3b29-46d7-a95b-58b8233b172d admin admin] RBAC: Authorizing `identity:delete_project(project_id=59c03b73d33b4a1cab20042f3a9729aa)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:10.373352 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace99169-3b29-46d7-a95b-58b8233b172d admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:10.421234 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-ace99169-3b29-46d7-a95b-58b8233b172d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59c03b73d33b4a1cab20042f3a9729aa', 'request_id': 'req-ace99169-3b29-46d7-a95b-58b8233b172d'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:10.428331 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-ace99169-3b29-46d7-a95b-58b8233b172d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59c03b73d33b4a1cab20042f3a9729aa', 'request_id': 'req-ace99169-3b29-46d7-a95b-58b8233b172d'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:10.433784 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-ace99169-3b29-46d7-a95b-58b8233b172d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59c03b73d33b4a1cab20042f3a9729aa', 'request_id': 'req-ace99169-3b29-46d7-a95b-58b8233b172d'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:10.444665 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-ace99169-3b29-46d7-a95b-58b8233b172d admin admin] The token cache is being invalidate because project 59c03b73d33b4a1cab20042f3a9729aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:20:10.444961 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-ace99169-3b29-46d7-a95b-58b8233b172d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:10.445591 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-ace99169-3b29-46d7-a95b-58b8233b172d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:10.447060 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1165/2322] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:20:10 2022] DELETE /identity/v3/projects/59c03b73d33b4a1cab20042f3a9729aa => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:10.451305 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa053a-a0a8-432c-a98f-11aa1b3d56f9 service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:10.460916 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa053a-a0a8-432c-a98f-11aa1b3d56f9 service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:10.521012 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbfefb-28f6-4871-bbfe-f9c2cf5e6586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:10.521650 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbfefb-28f6-4871-bbfe-f9c2cf5e6586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:10.521933 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbfefb-28f6-4871-bbfe-f9c2cf5e6586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:10.522211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbfefb-28f6-4871-bbfe-f9c2cf5e6586 admin admin] PATH_INFO: `/v3/users/e54e365ba71a4d0b8e99288c34c64180` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:10.523366 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbfefb-28f6-4871-bbfe-f9c2cf5e6586 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e54e365ba71a4d0b8e99288c34c64180)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:10.525390 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbfefb-28f6-4871-bbfe-f9c2cf5e6586 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:10.577411 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-29dbfefb-28f6-4871-bbfe-f9c2cf5e6586 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e54e365ba71a4d0b8e99288c34c64180'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:10.585216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-29dbfefb-28f6-4871-bbfe-f9c2cf5e6586 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e54e365ba71a4d0b8e99288c34c64180'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:10.599007 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-29dbfefb-28f6-4871-bbfe-f9c2cf5e6586 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e54e365ba71a4d0b8e99288c34c64180'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:10.599738 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-29dbfefb-28f6-4871-bbfe-f9c2cf5e6586 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e54e365ba71a4d0b8e99288c34c64180'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:10.600349 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-29dbfefb-28f6-4871-bbfe-f9c2cf5e6586 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e54e365ba71a4d0b8e99288c34c64180'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:10.617843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1158/2323] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:20:10 2022] DELETE /identity/v3/users/e54e365ba71a4d0b8e99288c34c64180 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:10.628103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace99169-3b29-46d7-a95b-58b8233b172d admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:10.629800 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ace99169-3b29-46d7-a95b-58b8233b172d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:10.644000 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace99169-3b29-46d7-a95b-58b8233b172d admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:10.725331 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e1a2a-6447-41c0-9855-192b146c9d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:10.726093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e1a2a-6447-41c0-9855-192b146c9d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:10.726560 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e1a2a-6447-41c0-9855-192b146c9d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:10.726679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e1a2a-6447-41c0-9855-192b146c9d44 admin admin] PATH_INFO: `/v3/users/18ee112cfcf643578474a181979bea83` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:10.728062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e1a2a-6447-41c0-9855-192b146c9d44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18ee112cfcf643578474a181979bea83)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:10.730209 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e1a2a-6447-41c0-9855-192b146c9d44 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:10.798530 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b07e1a2a-6447-41c0-9855-192b146c9d44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18ee112cfcf643578474a181979bea83'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:10.799512 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b07e1a2a-6447-41c0-9855-192b146c9d44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18ee112cfcf643578474a181979bea83'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:10.809578 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b07e1a2a-6447-41c0-9855-192b146c9d44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18ee112cfcf643578474a181979bea83'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:10.823458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:10.825460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b07e1a2a-6447-41c0-9855-192b146c9d44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18ee112cfcf643578474a181979bea83'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:10.836090 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b07e1a2a-6447-41c0-9855-192b146c9d44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18ee112cfcf643578474a181979bea83'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:10.844124 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1166/2324] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:20:10 2022] DELETE /identity/v3/users/18ee112cfcf643578474a181979bea83 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:10.848858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5b4f0-3959-480f-bf1e-bfd493c7176d None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:10.849495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5b4f0-3959-480f-bf1e-bfd493c7176d None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:10.850135 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5b4f0-3959-480f-bf1e-bfd493c7176d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:10.876223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-d9c5b4f0-3959-480f-bf1e-bfd493c7176d None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:10.877594 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d9c5b4f0-3959-480f-bf1e-bfd493c7176d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:10.911089 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:10.939109 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e1a2a-6447-41c0-9855-192b146c9d44 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:10.940578 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b07e1a2a-6447-41c0-9855-192b146c9d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:10.974752 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1159/2325] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:10.977555 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5b4f0-3959-480f-bf1e-bfd493c7176d None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:10.978999 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d9c5b4f0-3959-480f-bf1e-bfd493c7176d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:11.024842 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e1a2a-6447-41c0-9855-192b146c9d44 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:11.031243 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253106d-6ca0-4bd7-95b0-84d152519ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:11.031888 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253106d-6ca0-4bd7-95b0-84d152519ab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:11.032168 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253106d-6ca0-4bd7-95b0-84d152519ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:11.032440 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253106d-6ca0-4bd7-95b0-84d152519ab2 admin admin] PATH_INFO: `/v3/projects/d23f3f29f8834a59b885c8e8359fefb8` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:11.033620 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5253106d-6ca0-4bd7-95b0-84d152519ab2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d23f3f29f8834a59b885c8e8359fefb8)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:11.035564 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5253106d-6ca0-4bd7-95b0-84d152519ab2 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:11.051243 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5b4f0-3959-480f-bf1e-bfd493c7176d None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:11.055312 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9093de88-2cac-4443-967f-bb9637261997 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:11.056013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093de88-2cac-4443-967f-bb9637261997 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:11.056364 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093de88-2cac-4443-967f-bb9637261997 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:11.056760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9093de88-2cac-4443-967f-bb9637261997 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:11.066972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9093de88-2cac-4443-967f-bb9637261997 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:11.068605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9093de88-2cac-4443-967f-bb9637261997 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:11.082471 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1160/2326] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:20:11.089080 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5253106d-6ca0-4bd7-95b0-84d152519ab2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd23f3f29f8834a59b885c8e8359fefb8', 'request_id': 'req-5253106d-6ca0-4bd7-95b0-84d152519ab2'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:11.107141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5253106d-6ca0-4bd7-95b0-84d152519ab2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd23f3f29f8834a59b885c8e8359fefb8', 'request_id': 'req-5253106d-6ca0-4bd7-95b0-84d152519ab2'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:11.121436 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5253106d-6ca0-4bd7-95b0-84d152519ab2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd23f3f29f8834a59b885c8e8359fefb8', 'request_id': 'req-5253106d-6ca0-4bd7-95b0-84d152519ab2'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:11.136477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5253106d-6ca0-4bd7-95b0-84d152519ab2 admin admin] The token cache is being invalidate because project d23f3f29f8834a59b885c8e8359fefb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:20:11.136948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5253106d-6ca0-4bd7-95b0-84d152519ab2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:11.138203 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5253106d-6ca0-4bd7-95b0-84d152519ab2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:11.144748 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1167/2327] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:20:10 2022] DELETE /identity/v3/projects/d23f3f29f8834a59b885c8e8359fefb8 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:20:11.147138 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c79ad0-9dff-41ec-a1d4-d955dfcc76e2 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:11.147505 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c79ad0-9dff-41ec-a1d4-d955dfcc76e2 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:11.147788 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c79ad0-9dff-41ec-a1d4-d955dfcc76e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:11.177783 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-b0c79ad0-9dff-41ec-a1d4-d955dfcc76e2 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:11.179572 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b0c79ad0-9dff-41ec-a1d4-d955dfcc76e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:11.266151 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1161/2328] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:20:11.267511 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4e523-ec79-4cce-906c-9434528ce956 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:11.267803 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4e523-ec79-4cce-906c-9434528ce956 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:11.268084 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4e523-ec79-4cce-906c-9434528ce956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:11.288776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-2fb4e523-ec79-4cce-906c-9434528ce956 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:11.290022 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2fb4e523-ec79-4cce-906c-9434528ce956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:11.298559 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1168/2329] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:20:11.302784 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c79ad0-9dff-41ec-a1d4-d955dfcc76e2 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:11.313296 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c79ad0-9dff-41ec-a1d4-d955dfcc76e2 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:11.319015 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1827a2-4a20-422e-ac42-b47a6d8e58d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:11.319625 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1827a2-4a20-422e-ac42-b47a6d8e58d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:11.320016 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1827a2-4a20-422e-ac42-b47a6d8e58d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:11.320297 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1827a2-4a20-422e-ac42-b47a6d8e58d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:11.321104 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1827a2-4a20-422e-ac42-b47a6d8e58d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:20:11.321415 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1827a2-4a20-422e-ac42-b47a6d8e58d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:11.323113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1827a2-4a20-422e-ac42-b47a6d8e58d2 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:11.338601 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1162/2330] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:20:11.339761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4e523-ec79-4cce-906c-9434528ce956 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:11.349360 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4e523-ec79-4cce-906c-9434528ce956 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:11.354066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b20ce-5ab2-4574-9f7b-89975265ed10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:11.354661 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b20ce-5ab2-4574-9f7b-89975265ed10 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:11.354938 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b20ce-5ab2-4574-9f7b-89975265ed10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:11.355213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b20ce-5ab2-4574-9f7b-89975265ed10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:11.355847 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b20ce-5ab2-4574-9f7b-89975265ed10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:11.358766 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b20ce-5ab2-4574-9f7b-89975265ed10 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:11.386512 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1169/2331] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:20:11 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:20:11.389746 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1827a2-4a20-422e-ac42-b47a6d8e58d2 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:11.390486 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b20ce-5ab2-4574-9f7b-89975265ed10 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:11.391722 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-3c1827a2-4a20-422e-ac42-b47a6d8e58d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:11.405721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1827a2-4a20-422e-ac42-b47a6d8e58d2 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:11.442430 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:11.457184 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b20ce-5ab2-4574-9f7b-89975265ed10 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:11.462460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0d64c-a7f2-410f-8b40-6783dcea7a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:11.463060 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0d64c-a7f2-410f-8b40-6783dcea7a6e admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:11.463340 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0d64c-a7f2-410f-8b40-6783dcea7a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:11.463771 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0d64c-a7f2-410f-8b40-6783dcea7a6e admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:11.464395 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0d64c-a7f2-410f-8b40-6783dcea7a6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:11.466886 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0d64c-a7f2-410f-8b40-6783dcea7a6e admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:11.480257 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0b263-781e-4317-bdbf-da9c4e05d850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:11.480919 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0b263-781e-4317-bdbf-da9c4e05d850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:11.481200 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0b263-781e-4317-bdbf-da9c4e05d850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:11.481476 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0b263-781e-4317-bdbf-da9c4e05d850 admin admin] PATH_INFO: `/v3/projects/e4d4de8729614585818cdb9d325c54c2` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:11.482796 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0b263-781e-4317-bdbf-da9c4e05d850 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4d4de8729614585818cdb9d325c54c2)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:11.484727 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0b263-781e-4317-bdbf-da9c4e05d850 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:11.492004 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1170/2332] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:20:11 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:20:11.492212 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=100 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:11.493571 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0d64c-a7f2-410f-8b40-6783dcea7a6e admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:11.503454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0d64c-a7f2-410f-8b40-6783dcea7a6e admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:11.508075 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a369f-f22b-4176-8810-b01c9b8c8eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:11.508726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a369f-f22b-4176-8810-b01c9b8c8eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:11.508990 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a369f-f22b-4176-8810-b01c9b8c8eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:11.509267 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a369f-f22b-4176-8810-b01c9b8c8eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:11.509902 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a369f-f22b-4176-8810-b01c9b8c8eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:11.511590 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a369f-f22b-4176-8810-b01c9b8c8eb7 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:11.523307 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1171/2333] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:20:11.527192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a369f-f22b-4176-8810-b01c9b8c8eb7 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:11.534679 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-71c0b263-781e-4317-bdbf-da9c4e05d850 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4d4de8729614585818cdb9d325c54c2', 'request_id': 'req-71c0b263-781e-4317-bdbf-da9c4e05d850'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:11.537518 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a369f-f22b-4176-8810-b01c9b8c8eb7 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:11.542311 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-71c0b263-781e-4317-bdbf-da9c4e05d850 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4d4de8729614585818cdb9d325c54c2', 'request_id': 'req-71c0b263-781e-4317-bdbf-da9c4e05d850'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:11.550721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-71c0b263-781e-4317-bdbf-da9c4e05d850 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4d4de8729614585818cdb9d325c54c2', 'request_id': 'req-71c0b263-781e-4317-bdbf-da9c4e05d850'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:11.563671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-71c0b263-781e-4317-bdbf-da9c4e05d850 admin admin] The token cache is being invalidate because project e4d4de8729614585818cdb9d325c54c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:20:11.563965 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-71c0b263-781e-4317-bdbf-da9c4e05d850 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:11.564663 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-71c0b263-781e-4317-bdbf-da9c4e05d850 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:11.566219 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1163/2334] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:20:11 2022] DELETE /identity/v3/projects/e4d4de8729614585818cdb9d325c54c2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:11.570808 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50dfeb-d84f-48fc-bb87-d8738f3efa2f None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:11.571118 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50dfeb-d84f-48fc-bb87-d8738f3efa2f None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:11.571396 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50dfeb-d84f-48fc-bb87-d8738f3efa2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:11.592050 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-ad50dfeb-d84f-48fc-bb87-d8738f3efa2f None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:11.593469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ad50dfeb-d84f-48fc-bb87-d8738f3efa2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:11.606903 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b98f3-1446-477b-9e42-f94deecd37cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:11.607654 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b98f3-1446-477b-9e42-f94deecd37cf admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:11.607941 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b98f3-1446-477b-9e42-f94deecd37cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:11.608219 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b98f3-1446-477b-9e42-f94deecd37cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:11.608987 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1b98f3-1446-477b-9e42-f94deecd37cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:11.610578 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1b98f3-1446-477b-9e42-f94deecd37cf admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:11.621516 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1172/2335] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:20:11.626417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b98f3-1446-477b-9e42-f94deecd37cf admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:11.628094 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-4d1b98f3-1446-477b-9e42-f94deecd37cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:11.641633 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b98f3-1446-477b-9e42-f94deecd37cf admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:11.646394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4705d-12f0-4d06-bb12-f29e6fdd73b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:11.647042 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4705d-12f0-4d06-bb12-f29e6fdd73b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:11.647326 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4705d-12f0-4d06-bb12-f29e6fdd73b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:11.647603 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4705d-12f0-4d06-bb12-f29e6fdd73b8 admin admin] PATH_INFO: `/v3/projects/5dc246ad81ed43a0b7170adc05e047f4/users/d17ddde063034ef9935c0dd5b55666b7/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:11.658253 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1164/2336] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:20:11.662615 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4705d-12f0-4d06-bb12-f29e6fdd73b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dc246ad81ed43a0b7170adc05e047f4, user_id=d17ddde063034ef9935c0dd5b55666b7, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:11.665285 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f4705d-12f0-4d06-bb12-f29e6fdd73b8 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:11.665933 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84bef0-0d9f-43ec-8bd4-b6afac4fbf35 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:11.666225 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84bef0-0d9f-43ec-8bd4-b6afac4fbf35 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:11.666625 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84bef0-0d9f-43ec-8bd4-b6afac4fbf35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:11.679058 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1173/2337] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:20:11 2022] PUT /identity/v3/projects/5dc246ad81ed43a0b7170adc05e047f4/users/d17ddde063034ef9935c0dd5b55666b7/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:11.684023 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307e45d-c4ec-483b-aba4-fc0d43ce5724 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:11.684301 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307e45d-c4ec-483b-aba4-fc0d43ce5724 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:11.684614 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307e45d-c4ec-483b-aba4-fc0d43ce5724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:11.689060 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-3b84bef0-0d9f-43ec-8bd4-b6afac4fbf35 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:11.690278 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-3b84bef0-0d9f-43ec-8bd4-b6afac4fbf35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:11.704895 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-2307e45d-c4ec-483b-aba4-fc0d43ce5724 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:11.706083 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2307e45d-c4ec-483b-aba4-fc0d43ce5724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:11.751272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1165/2338] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:20:11.755440 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84bef0-0d9f-43ec-8bd4-b6afac4fbf35 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:11.764899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84bef0-0d9f-43ec-8bd4-b6afac4fbf35 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:11.765633 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1174/2339] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:20:11.769530 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b7fd7-95c6-47e8-a8aa-69fb5da4f97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:11.770130 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b7fd7-95c6-47e8-a8aa-69fb5da4f97f admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:11.770414 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b7fd7-95c6-47e8-a8aa-69fb5da4f97f admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:11.770691 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b7fd7-95c6-47e8-a8aa-69fb5da4f97f admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:11.771333 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b7fd7-95c6-47e8-a8aa-69fb5da4f97f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:20:11.771635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b7fd7-95c6-47e8-a8aa-69fb5da4f97f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:11.773269 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b7fd7-95c6-47e8-a8aa-69fb5da4f97f admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:11.774953 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307e45d-c4ec-483b-aba4-fc0d43ce5724 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:11.776396 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2307e45d-c4ec-483b-aba4-fc0d43ce5724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:11.788108 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1166/2340] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:20:11.792550 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b7fd7-95c6-47e8-a8aa-69fb5da4f97f admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:11.803497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b7fd7-95c6-47e8-a8aa-69fb5da4f97f admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:11.809394 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0182d64-c583-4fb2-83c4-b3ee1ae9bfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:11.810859 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0182d64-c583-4fb2-83c4-b3ee1ae9bfe2 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:11.811474 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0182d64-c583-4fb2-83c4-b3ee1ae9bfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:11.811474 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0182d64-c583-4fb2-83c4-b3ee1ae9bfe2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:11.812184 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0182d64-c583-4fb2-83c4-b3ee1ae9bfe2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:11.815016 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0182d64-c583-4fb2-83c4-b3ee1ae9bfe2 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:11.827324 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:11.843874 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307e45d-c4ec-483b-aba4-fc0d43ce5724 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:11.844927 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:11.847979 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43b67f-45d5-42a8-b9eb-6daa887e4491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:11.848753 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43b67f-45d5-42a8-b9eb-6daa887e4491 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:11.849154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43b67f-45d5-42a8-b9eb-6daa887e4491 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:11.849435 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43b67f-45d5-42a8-b9eb-6daa887e4491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:11.857884 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1167/2341] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:20:11 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:20:11.858903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0182d64-c583-4fb2-83c4-b3ee1ae9bfe2 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:11.861437 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43b67f-45d5-42a8-b9eb-6daa887e4491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:11.862915 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43b67f-45d5-42a8-b9eb-6daa887e4491 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:11.876231 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1175/2342] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:20:11.925664 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0182d64-c583-4fb2-83c4-b3ee1ae9bfe2 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:11.931934 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d2dddd-491d-4bb5-a48f-c91355109025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:11.932950 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2dddd-491d-4bb5-a48f-c91355109025 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:11.933234 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2dddd-491d-4bb5-a48f-c91355109025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:11.933519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d2dddd-491d-4bb5-a48f-c91355109025 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:11.934158 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d2dddd-491d-4bb5-a48f-c91355109025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:11.937486 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d2dddd-491d-4bb5-a48f-c91355109025 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:11.962015 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1168/2343] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:20:11 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:20:11.967104 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43b67f-45d5-42a8-b9eb-6daa887e4491 service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:11.977201 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43b67f-45d5-42a8-b9eb-6daa887e4491 service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:11.982840 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ffba1-3944-4016-bf7c-b78f17fb5dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:11.983478 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ffba1-3944-4016-bf7c-b78f17fb5dec admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:11.983757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ffba1-3944-4016-bf7c-b78f17fb5dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:11.984052 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ffba1-3944-4016-bf7c-b78f17fb5dec admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:11.984717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ffba1-3944-4016-bf7c-b78f17fb5dec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:11.986345 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ffba1-3944-4016-bf7c-b78f17fb5dec admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:11.996706 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1176/2344] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:20:12.001381 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d2dddd-491d-4bb5-a48f-c91355109025 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:12.012282 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d2dddd-491d-4bb5-a48f-c91355109025 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:12.017408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37035cf6-ff59-42a9-a0df-398e32325352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:12.018186 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37035cf6-ff59-42a9-a0df-398e32325352 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:12.018619 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37035cf6-ff59-42a9-a0df-398e32325352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:12.018972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37035cf6-ff59-42a9-a0df-398e32325352 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:12.019689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37035cf6-ff59-42a9-a0df-398e32325352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:12.021466 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37035cf6-ff59-42a9-a0df-398e32325352 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:12.032303 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1169/2345] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:20:12.036831 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ffba1-3944-4016-bf7c-b78f17fb5dec admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:12.047839 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ffba1-3944-4016-bf7c-b78f17fb5dec admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:12.053547 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563c67e-a7b1-4be6-88bb-c3ed4641c118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:12.054712 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563c67e-a7b1-4be6-88bb-c3ed4641c118 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:12.054712 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563c67e-a7b1-4be6-88bb-c3ed4641c118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:12.055091 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563c67e-a7b1-4be6-88bb-c3ed4641c118 admin admin] PATH_INFO: `/v3/projects/6070d88ec7554a4ab2be3d0e74483a4e/users/af07bd5cd97f4a908503734110dde8fb/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:12.071837 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563c67e-a7b1-4be6-88bb-c3ed4641c118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6070d88ec7554a4ab2be3d0e74483a4e, user_id=af07bd5cd97f4a908503734110dde8fb, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:12.075475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563c67e-a7b1-4be6-88bb-c3ed4641c118 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:12.089099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1177/2346] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:20:12 2022] PUT /identity/v3/projects/6070d88ec7554a4ab2be3d0e74483a4e/users/af07bd5cd97f4a908503734110dde8fb/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:12.101513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65171c6-6616-4fe1-8359-2b34141cb500 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:12.101800 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65171c6-6616-4fe1-8359-2b34141cb500 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:12.102073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65171c6-6616-4fe1-8359-2b34141cb500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:12.134705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-b65171c6-6616-4fe1-8359-2b34141cb500 None None] MFA Rules not processed for user `af07bd5cd97f4a908503734110dde8fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:12.136046 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b65171c6-6616-4fe1-8359-2b34141cb500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:12.211572 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1170/2347] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:12.224434 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563c67e-a7b1-4be6-88bb-c3ed4641c118 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:12.226348 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-5563c67e-a7b1-4be6-88bb-c3ed4641c118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:12.293632 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563c67e-a7b1-4be6-88bb-c3ed4641c118 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:12.298143 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947c6c3-4e9d-4342-961f-efaee3fcefa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:12.299572 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947c6c3-4e9d-4342-961f-efaee3fcefa9 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:12.299719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947c6c3-4e9d-4342-961f-efaee3fcefa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:12.299947 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947c6c3-4e9d-4342-961f-efaee3fcefa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:12.310861 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947c6c3-4e9d-4342-961f-efaee3fcefa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:12.312359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947c6c3-4e9d-4342-961f-efaee3fcefa9 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:12.321911 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1178/2348] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:20:12.546270 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa87ed-4712-48bf-9cea-589d67b75372 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:12.546489 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa87ed-4712-48bf-9cea-589d67b75372 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:12.546742 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa87ed-4712-48bf-9cea-589d67b75372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:12.576320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-e7fa87ed-4712-48bf-9cea-589d67b75372 None None] MFA Rules not processed for user `d17ddde063034ef9935c0dd5b55666b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:12.577690 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e7fa87ed-4712-48bf-9cea-589d67b75372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:12.671044 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:12.681157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1171/2349] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:20:12.686199 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947c6c3-4e9d-4342-961f-efaee3fcefa9 service cinder] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:12.756408 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947c6c3-4e9d-4342-961f-efaee3fcefa9 service cinder] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:12.761292 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418596b-ef28-4ea4-8a01-7bbe7be9a08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:12.761904 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418596b-ef28-4ea4-8a01-7bbe7be9a08b admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:12.762194 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418596b-ef28-4ea4-8a01-7bbe7be9a08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:12.762476 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418596b-ef28-4ea4-8a01-7bbe7be9a08b admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:12.763682 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418596b-ef28-4ea4-8a01-7bbe7be9a08b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:12.765814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418596b-ef28-4ea4-8a01-7bbe7be9a08b admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:12.791763 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1179/2350] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:20:12 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:20:12.795836 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa87ed-4712-48bf-9cea-589d67b75372 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:12.806573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa87ed-4712-48bf-9cea-589d67b75372 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:12.870321 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d94cd2-a2dc-470b-92bf-2089bc9359e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:12.871034 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d94cd2-a2dc-470b-92bf-2089bc9359e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:12.871208 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d94cd2-a2dc-470b-92bf-2089bc9359e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:12.871490 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d94cd2-a2dc-470b-92bf-2089bc9359e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:12.872126 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d94cd2-a2dc-470b-92bf-2089bc9359e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:12.874330 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d94cd2-a2dc-470b-92bf-2089bc9359e8 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:12.903287 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1172/2351] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:20:12 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:20:12.908080 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418596b-ef28-4ea4-8a01-7bbe7be9a08b admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:12.918408 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418596b-ef28-4ea4-8a01-7bbe7be9a08b admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:12.923056 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533bff2a-6533-4ebf-b0cb-070420bf54b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:12.923653 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bff2a-6533-4ebf-b0cb-070420bf54b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:12.923937 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bff2a-6533-4ebf-b0cb-070420bf54b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:12.924215 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bff2a-6533-4ebf-b0cb-070420bf54b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:12.924889 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533bff2a-6533-4ebf-b0cb-070420bf54b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:12.926476 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533bff2a-6533-4ebf-b0cb-070420bf54b4 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:12.937130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1180/2352] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:20:12.941337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d94cd2-a2dc-470b-92bf-2089bc9359e8 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:12.951697 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d94cd2-a2dc-470b-92bf-2089bc9359e8 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:12.956223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19994a8-9184-44f2-96b5-5678b1c57fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:12.956904 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19994a8-9184-44f2-96b5-5678b1c57fdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:12.957214 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19994a8-9184-44f2-96b5-5678b1c57fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:12.957494 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19994a8-9184-44f2-96b5-5678b1c57fdf admin admin] PATH_INFO: `/v3/projects/104e33ddffdc49af9baa6ecda4d054d5/users/8cd15bbfa7e64810a6cef86ae58acd8c/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:12.972378 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19994a8-9184-44f2-96b5-5678b1c57fdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=104e33ddffdc49af9baa6ecda4d054d5, user_id=8cd15bbfa7e64810a6cef86ae58acd8c, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:12.975048 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19994a8-9184-44f2-96b5-5678b1c57fdf admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:12.988369 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1173/2353] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:20:12 2022] PUT /identity/v3/projects/104e33ddffdc49af9baa6ecda4d054d5/users/8cd15bbfa7e64810a6cef86ae58acd8c/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:12.992580 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533bff2a-6533-4ebf-b0cb-070420bf54b4 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:13.002468 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533bff2a-6533-4ebf-b0cb-070420bf54b4 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:13.021923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:13.058075 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9894c4-31ed-459b-b5cc-343169eca423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:13.058599 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9894c4-31ed-459b-b5cc-343169eca423 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:13.058879 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9894c4-31ed-459b-b5cc-343169eca423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:13.059151 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9894c4-31ed-459b-b5cc-343169eca423 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:13.059784 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9894c4-31ed-459b-b5cc-343169eca423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:13.061418 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9894c4-31ed-459b-b5cc-343169eca423 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:13.070809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1181/2354] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:20:13.075209 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19994a8-9184-44f2-96b5-5678b1c57fdf admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:13.086216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19994a8-9184-44f2-96b5-5678b1c57fdf admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:13.090768 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60fd12-b5df-4085-8fe8-9696db738b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:13.092131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60fd12-b5df-4085-8fe8-9696db738b82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:13.092439 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60fd12-b5df-4085-8fe8-9696db738b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:13.092713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60fd12-b5df-4085-8fe8-9696db738b82 admin admin] PATH_INFO: `/v3/projects/104e33ddffdc49af9baa6ecda4d054d5/users/8cd15bbfa7e64810a6cef86ae58acd8c/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:13.094628 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd60fd12-b5df-4085-8fe8-9696db738b82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=104e33ddffdc49af9baa6ecda4d054d5, user_id=8cd15bbfa7e64810a6cef86ae58acd8c, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:13.097244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd60fd12-b5df-4085-8fe8-9696db738b82 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:13.107650 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1174/2355] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:20:13 2022] PUT /identity/v3/projects/104e33ddffdc49af9baa6ecda4d054d5/users/8cd15bbfa7e64810a6cef86ae58acd8c/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:13.743896 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c9144-a44c-4dbe-928e-bdca1460afdd None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:13.744096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c9144-a44c-4dbe-928e-bdca1460afdd None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:13.744379 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c9144-a44c-4dbe-928e-bdca1460afdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:13.781519 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-8c9c9144-a44c-4dbe-928e-bdca1460afdd None None] MFA Rules not processed for user `8cd15bbfa7e64810a6cef86ae58acd8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:13.782803 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8c9c9144-a44c-4dbe-928e-bdca1460afdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:13.864639 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1182/2356] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:13.878518 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60fd12-b5df-4085-8fe8-9696db738b82 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:13.940592 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60fd12-b5df-4085-8fe8-9696db738b82 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:13.944187 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf754ba-f0c3-4c21-adf0-330215ba5688 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:13.944833 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf754ba-f0c3-4c21-adf0-330215ba5688 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:13.945130 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf754ba-f0c3-4c21-adf0-330215ba5688 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:13.945400 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf754ba-f0c3-4c21-adf0-330215ba5688 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:14.006321 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf754ba-f0c3-4c21-adf0-330215ba5688 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:14.007743 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf754ba-f0c3-4c21-adf0-330215ba5688 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:14.017304 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1175/2357] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:20:15.401509 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c9144-a44c-4dbe-928e-bdca1460afdd None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:15.403086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8c9c9144-a44c-4dbe-928e-bdca1460afdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:15.415949 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c9144-a44c-4dbe-928e-bdca1460afdd None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:15.420004 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610e081-a94f-46f0-886d-6c2aa7759f12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:15.421173 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610e081-a94f-46f0-886d-6c2aa7759f12 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:15.421451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610e081-a94f-46f0-886d-6c2aa7759f12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:15.421723 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610e081-a94f-46f0-886d-6c2aa7759f12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:15.423502 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf754ba-f0c3-4c21-adf0-330215ba5688 service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:15.428688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:15.431475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610e081-a94f-46f0-886d-6c2aa7759f12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:15.432976 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610e081-a94f-46f0-886d-6c2aa7759f12 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:15.443107 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1183/2358] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:20:15.494963 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf754ba-f0c3-4c21-adf0-330215ba5688 service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:15.500860 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186a628-a2f2-4698-b411-bf81b707e078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:15.501495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186a628-a2f2-4698-b411-bf81b707e078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:15.501778 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186a628-a2f2-4698-b411-bf81b707e078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:15.502061 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186a628-a2f2-4698-b411-bf81b707e078 admin admin] PATH_INFO: `/v3/users/af07bd5cd97f4a908503734110dde8fb` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:15.503420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186a628-a2f2-4698-b411-bf81b707e078 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af07bd5cd97f4a908503734110dde8fb)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:15.505297 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186a628-a2f2-4698-b411-bf81b707e078 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:15.561139 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d186a628-a2f2-4698-b411-bf81b707e078 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af07bd5cd97f4a908503734110dde8fb'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:15.568497 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d186a628-a2f2-4698-b411-bf81b707e078 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af07bd5cd97f4a908503734110dde8fb'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:15.580722 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d186a628-a2f2-4698-b411-bf81b707e078 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af07bd5cd97f4a908503734110dde8fb'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:15.581657 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d186a628-a2f2-4698-b411-bf81b707e078 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af07bd5cd97f4a908503734110dde8fb'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:15.582463 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d186a628-a2f2-4698-b411-bf81b707e078 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af07bd5cd97f4a908503734110dde8fb'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:15.596144 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1176/2359] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:20:15 2022] DELETE /identity/v3/users/af07bd5cd97f4a908503734110dde8fb => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:20:15.600986 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4501e-6745-40a5-863a-cea47def84ca None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:15.601378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4501e-6745-40a5-863a-cea47def84ca None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:15.601679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4501e-6745-40a5-863a-cea47def84ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:15.625420 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-3dc4501e-6745-40a5-863a-cea47def84ca None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:15.626665 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-3dc4501e-6745-40a5-863a-cea47def84ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:15.689466 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1184/2360] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:15.699343 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186a628-a2f2-4698-b411-bf81b707e078 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:15.700842 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-d186a628-a2f2-4698-b411-bf81b707e078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:15.760013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186a628-a2f2-4698-b411-bf81b707e078 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:15.764684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30457825-058d-4691-b448-2ba79b6cfa45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:15.765326 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30457825-058d-4691-b448-2ba79b6cfa45 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:15.765615 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30457825-058d-4691-b448-2ba79b6cfa45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:15.765927 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30457825-058d-4691-b448-2ba79b6cfa45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:15.775113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30457825-058d-4691-b448-2ba79b6cfa45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:15.776664 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30457825-058d-4691-b448-2ba79b6cfa45 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:15.787586 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1177/2361] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:20:16.059576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc4501e-6745-40a5-863a-cea47def84ca None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:16.061669 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-3dc4501e-6745-40a5-863a-cea47def84ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:16.073664 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc4501e-6745-40a5-863a-cea47def84ca None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:16.078911 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6f9ee-729c-4d7d-b762-a624f39055bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:16.079547 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6f9ee-729c-4d7d-b762-a624f39055bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:16.079827 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6f9ee-729c-4d7d-b762-a624f39055bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:16.080104 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6f9ee-729c-4d7d-b762-a624f39055bc admin admin] PATH_INFO: `/v3/projects/6070d88ec7554a4ab2be3d0e74483a4e` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:16.081264 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6f9ee-729c-4d7d-b762-a624f39055bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6070d88ec7554a4ab2be3d0e74483a4e)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:16.083230 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6f9ee-729c-4d7d-b762-a624f39055bc admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:16.117292 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:16.132956 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b5a6f9ee-729c-4d7d-b762-a624f39055bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6070d88ec7554a4ab2be3d0e74483a4e', 'request_id': 'req-b5a6f9ee-729c-4d7d-b762-a624f39055bc'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:16.140055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b5a6f9ee-729c-4d7d-b762-a624f39055bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6070d88ec7554a4ab2be3d0e74483a4e', 'request_id': 'req-b5a6f9ee-729c-4d7d-b762-a624f39055bc'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:16.145570 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b5a6f9ee-729c-4d7d-b762-a624f39055bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6070d88ec7554a4ab2be3d0e74483a4e', 'request_id': 'req-b5a6f9ee-729c-4d7d-b762-a624f39055bc'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:16.157229 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b5a6f9ee-729c-4d7d-b762-a624f39055bc admin admin] The token cache is being invalidate because project 6070d88ec7554a4ab2be3d0e74483a4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:20:16.157517 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b5a6f9ee-729c-4d7d-b762-a624f39055bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:16.158151 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b5a6f9ee-729c-4d7d-b762-a624f39055bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:16.159627 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1185/2362] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:20:16 2022] DELETE /identity/v3/projects/6070d88ec7554a4ab2be3d0e74483a4e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:16.165467 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede490cf-a3fd-4c6d-8c29-806a854148df None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:16.165760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede490cf-a3fd-4c6d-8c29-806a854148df None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:16.166043 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede490cf-a3fd-4c6d-8c29-806a854148df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:16.187266 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-ede490cf-a3fd-4c6d-8c29-806a854148df None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:16.188461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ede490cf-a3fd-4c6d-8c29-806a854148df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:16.208722 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:16.248985 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1178/2363] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:16.256881 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e98ebe-fea3-48ed-8ba5-1a6cbb236b57 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:16.257174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e98ebe-fea3-48ed-8ba5-1a6cbb236b57 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:16.257453 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e98ebe-fea3-48ed-8ba5-1a6cbb236b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:16.276764 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-51e98ebe-fea3-48ed-8ba5-1a6cbb236b57 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:16.277966 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-51e98ebe-fea3-48ed-8ba5-1a6cbb236b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:16.286642 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1186/2364] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:16.291296 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede490cf-a3fd-4c6d-8c29-806a854148df None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:16.302131 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede490cf-a3fd-4c6d-8c29-806a854148df None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:16.307419 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63617f-7615-465b-9e02-73f75ce8407e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:16.308076 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63617f-7615-465b-9e02-73f75ce8407e admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:16.308320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63617f-7615-465b-9e02-73f75ce8407e admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:16.308848 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63617f-7615-465b-9e02-73f75ce8407e admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:16.309536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63617f-7615-465b-9e02-73f75ce8407e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:20:16.309841 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63617f-7615-465b-9e02-73f75ce8407e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:16.311618 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63617f-7615-465b-9e02-73f75ce8407e admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:16.323366 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1179/2365] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:20:16.327948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e98ebe-fea3-48ed-8ba5-1a6cbb236b57 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:16.337198 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e98ebe-fea3-48ed-8ba5-1a6cbb236b57 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:16.342618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b40c518-dd1b-4418-9b43-dae29f91eaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:16.343214 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b40c518-dd1b-4418-9b43-dae29f91eaa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:16.343495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b40c518-dd1b-4418-9b43-dae29f91eaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:16.343768 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b40c518-dd1b-4418-9b43-dae29f91eaa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:16.344394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b40c518-dd1b-4418-9b43-dae29f91eaa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:16.346304 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b40c518-dd1b-4418-9b43-dae29f91eaa8 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:16.374333 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1187/2366] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:20:16 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:20:16.378468 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63617f-7615-465b-9e02-73f75ce8407e admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:16.388917 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63617f-7615-465b-9e02-73f75ce8407e admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:16.444623 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffd909-a349-4b5b-9468-2301d07f2628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:16.445233 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffd909-a349-4b5b-9468-2301d07f2628 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:16.445345 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffd909-a349-4b5b-9468-2301d07f2628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:16.445620 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffd909-a349-4b5b-9468-2301d07f2628 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:16.446241 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ffd909-a349-4b5b-9468-2301d07f2628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:16.448115 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ffd909-a349-4b5b-9468-2301d07f2628 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:16.471217 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1180/2367] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:20:16 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:20:16.475095 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b40c518-dd1b-4418-9b43-dae29f91eaa8 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:16.485753 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b40c518-dd1b-4418-9b43-dae29f91eaa8 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:16.490825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfd3f5-54c1-425e-abf9-75d8cbe75b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:16.491408 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfd3f5-54c1-425e-abf9-75d8cbe75b21 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:16.491688 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfd3f5-54c1-425e-abf9-75d8cbe75b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:16.491960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfd3f5-54c1-425e-abf9-75d8cbe75b21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:16.492625 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfd3f5-54c1-425e-abf9-75d8cbe75b21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:16.494207 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfd3f5-54c1-425e-abf9-75d8cbe75b21 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:16.503907 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1188/2368] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:20:16.507789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffd909-a349-4b5b-9468-2301d07f2628 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:16.518130 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffd909-a349-4b5b-9468-2301d07f2628 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:16.522751 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8cab2-ac4f-4874-bbee-ba9a9c97d98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:16.523332 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8cab2-ac4f-4874-bbee-ba9a9c97d98e admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:16.523612 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8cab2-ac4f-4874-bbee-ba9a9c97d98e admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:16.523887 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8cab2-ac4f-4874-bbee-ba9a9c97d98e admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:16.524539 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f8cab2-ac4f-4874-bbee-ba9a9c97d98e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:16.526112 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f8cab2-ac4f-4874-bbee-ba9a9c97d98e admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:16.537285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1181/2369] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:20:16.541206 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfd3f5-54c1-425e-abf9-75d8cbe75b21 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:16.551035 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfd3f5-54c1-425e-abf9-75d8cbe75b21 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:16.555606 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a7a6e-8eb8-47ca-bf16-4bdbbd41df58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:16.556247 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a7a6e-8eb8-47ca-bf16-4bdbbd41df58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:16.556564 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a7a6e-8eb8-47ca-bf16-4bdbbd41df58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:16.556848 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a7a6e-8eb8-47ca-bf16-4bdbbd41df58 admin admin] PATH_INFO: `/v3/projects/5ff1f3f17cd043468344cd9c7d1e2ffe/users/92fe16d2f8b3408c88bc2a14b0cf5591/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:16.572069 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a7a6e-8eb8-47ca-bf16-4bdbbd41df58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ff1f3f17cd043468344cd9c7d1e2ffe, user_id=92fe16d2f8b3408c88bc2a14b0cf5591, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:16.574697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a7a6e-8eb8-47ca-bf16-4bdbbd41df58 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:16.588149 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1189/2370] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:20:16 2022] PUT /identity/v3/projects/5ff1f3f17cd043468344cd9c7d1e2ffe/users/92fe16d2f8b3408c88bc2a14b0cf5591/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:16.599942 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368a1b4-56e8-4016-bd8e-9f8215981276 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:16.600643 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368a1b4-56e8-4016-bd8e-9f8215981276 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:16.601170 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368a1b4-56e8-4016-bd8e-9f8215981276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:16.631066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-9368a1b4-56e8-4016-bd8e-9f8215981276 None None] MFA Rules not processed for user `92fe16d2f8b3408c88bc2a14b0cf5591`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:16.632286 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9368a1b4-56e8-4016-bd8e-9f8215981276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:16.700095 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1182/2371] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:16.712255 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a7a6e-8eb8-47ca-bf16-4bdbbd41df58 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:16.714033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b47a7a6e-8eb8-47ca-bf16-4bdbbd41df58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:16.777772 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a7a6e-8eb8-47ca-bf16-4bdbbd41df58 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:16.782394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54aba8c-e34f-40e9-acb2-e90027308ab7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:16.783006 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54aba8c-e34f-40e9-acb2-e90027308ab7 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:16.783292 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54aba8c-e34f-40e9-acb2-e90027308ab7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:16.783607 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54aba8c-e34f-40e9-acb2-e90027308ab7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:16.792693 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54aba8c-e34f-40e9-acb2-e90027308ab7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:16.794125 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54aba8c-e34f-40e9-acb2-e90027308ab7 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:16.807068 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1190/2372] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:20:16.833338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368a1b4-56e8-4016-bd8e-9f8215981276 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:16.865398 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:16.917053 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368a1b4-56e8-4016-bd8e-9f8215981276 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:16.922680 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938890d4-683f-445a-a82c-7b54070a83fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:16.923385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938890d4-683f-445a-a82c-7b54070a83fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:16.923694 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938890d4-683f-445a-a82c-7b54070a83fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:16.924020 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938890d4-683f-445a-a82c-7b54070a83fc admin admin] PATH_INFO: `/v3/users/92fe16d2f8b3408c88bc2a14b0cf5591` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:16.925390 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938890d4-683f-445a-a82c-7b54070a83fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=92fe16d2f8b3408c88bc2a14b0cf5591)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:16.927427 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938890d4-683f-445a-a82c-7b54070a83fc admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:16.982997 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-938890d4-683f-445a-a82c-7b54070a83fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92fe16d2f8b3408c88bc2a14b0cf5591'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:16.994281 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-938890d4-683f-445a-a82c-7b54070a83fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92fe16d2f8b3408c88bc2a14b0cf5591'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:17.008338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-938890d4-683f-445a-a82c-7b54070a83fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92fe16d2f8b3408c88bc2a14b0cf5591'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:17.009113 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-938890d4-683f-445a-a82c-7b54070a83fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92fe16d2f8b3408c88bc2a14b0cf5591'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:17.009736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-938890d4-683f-445a-a82c-7b54070a83fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92fe16d2f8b3408c88bc2a14b0cf5591'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:17.028592 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1183/2373] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:20:16 2022] DELETE /identity/v3/users/92fe16d2f8b3408c88bc2a14b0cf5591 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:17.033852 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f26cf6-0856-4e6e-b1e3-7c6a8b23e8f4 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:17.034140 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f26cf6-0856-4e6e-b1e3-7c6a8b23e8f4 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:17.034415 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f26cf6-0856-4e6e-b1e3-7c6a8b23e8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:17.053846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-20f26cf6-0856-4e6e-b1e3-7c6a8b23e8f4 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:17.055055 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-20f26cf6-0856-4e6e-b1e3-7c6a8b23e8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:17.111335 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:17.126079 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1191/2374] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:17.135711 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938890d4-683f-445a-a82c-7b54070a83fc admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:17.137112 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-938890d4-683f-445a-a82c-7b54070a83fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:17.194399 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938890d4-683f-445a-a82c-7b54070a83fc admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:17.198030 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c984dc-1219-4352-95ab-5924456af0f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:17.198633 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c984dc-1219-4352-95ab-5924456af0f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:17.198914 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c984dc-1219-4352-95ab-5924456af0f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:17.199189 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c984dc-1219-4352-95ab-5924456af0f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:17.209455 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c984dc-1219-4352-95ab-5924456af0f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:17.210803 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c984dc-1219-4352-95ab-5924456af0f0 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:17.220581 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1184/2375] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:20:17.489863 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f26cf6-0856-4e6e-b1e3-7c6a8b23e8f4 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:17.491358 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-20f26cf6-0856-4e6e-b1e3-7c6a8b23e8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:17.505161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f26cf6-0856-4e6e-b1e3-7c6a8b23e8f4 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:17.509922 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a75db-b067-4843-8772-4cac3e6d6c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:17.510562 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a75db-b067-4843-8772-4cac3e6d6c3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:17.510846 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a75db-b067-4843-8772-4cac3e6d6c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:17.511121 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a75db-b067-4843-8772-4cac3e6d6c3c admin admin] PATH_INFO: `/v3/projects/5ff1f3f17cd043468344cd9c7d1e2ffe` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:17.512242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a75db-b067-4843-8772-4cac3e6d6c3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ff1f3f17cd043468344cd9c7d1e2ffe)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:17.514273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a75db-b067-4843-8772-4cac3e6d6c3c admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:17.560568 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-174a75db-b067-4843-8772-4cac3e6d6c3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ff1f3f17cd043468344cd9c7d1e2ffe', 'request_id': 'req-174a75db-b067-4843-8772-4cac3e6d6c3c'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:17.567906 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-174a75db-b067-4843-8772-4cac3e6d6c3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ff1f3f17cd043468344cd9c7d1e2ffe', 'request_id': 'req-174a75db-b067-4843-8772-4cac3e6d6c3c'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:17.573598 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-174a75db-b067-4843-8772-4cac3e6d6c3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ff1f3f17cd043468344cd9c7d1e2ffe', 'request_id': 'req-174a75db-b067-4843-8772-4cac3e6d6c3c'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:17.585331 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-174a75db-b067-4843-8772-4cac3e6d6c3c admin admin] The token cache is being invalidate because project 5ff1f3f17cd043468344cd9c7d1e2ffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:20:17.585623 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-174a75db-b067-4843-8772-4cac3e6d6c3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:17.586255 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-174a75db-b067-4843-8772-4cac3e6d6c3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:17.587742 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1192/2376] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:20:17 2022] DELETE /identity/v3/projects/5ff1f3f17cd043468344cd9c7d1e2ffe => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:17.593676 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c226fe8-a68b-414b-b620-2fe1c41a801f None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:17.594175 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c226fe8-a68b-414b-b620-2fe1c41a801f None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:17.594311 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c226fe8-a68b-414b-b620-2fe1c41a801f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:17.613308 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-8c226fe8-a68b-414b-b620-2fe1c41a801f None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:17.614752 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-8c226fe8-a68b-414b-b620-2fe1c41a801f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:17.655139 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:17.676039 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1185/2377] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:17.740600 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fafca7-34ee-4a4e-97c8-2737ba348fb8 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:17.740901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fafca7-34ee-4a4e-97c8-2737ba348fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:17.741088 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fafca7-34ee-4a4e-97c8-2737ba348fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:17.761102 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-c5fafca7-34ee-4a4e-97c8-2737ba348fb8 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:17.762305 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c5fafca7-34ee-4a4e-97c8-2737ba348fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:17.770833 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1193/2378] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:17.775040 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c226fe8-a68b-414b-b620-2fe1c41a801f None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:17.784938 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c226fe8-a68b-414b-b620-2fe1c41a801f None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:17.789534 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112af1fa-ab4d-4d06-9a06-d7dfc32f3da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:17.790129 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112af1fa-ab4d-4d06-9a06-d7dfc32f3da9 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:17.790411 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112af1fa-ab4d-4d06-9a06-d7dfc32f3da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:17.790716 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112af1fa-ab4d-4d06-9a06-d7dfc32f3da9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:17.791387 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112af1fa-ab4d-4d06-9a06-d7dfc32f3da9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:20:17.791693 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112af1fa-ab4d-4d06-9a06-d7dfc32f3da9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:17.793308 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112af1fa-ab4d-4d06-9a06-d7dfc32f3da9 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:17.804867 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1186/2379] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:20:17.809250 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fafca7-34ee-4a4e-97c8-2737ba348fb8 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:17.819273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fafca7-34ee-4a4e-97c8-2737ba348fb8 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:17.823918 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5beb5-3dc9-44fe-9ef5-d6e5d7ce469a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:17.824535 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5beb5-3dc9-44fe-9ef5-d6e5d7ce469a admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:17.824822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5beb5-3dc9-44fe-9ef5-d6e5d7ce469a admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:17.825101 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5beb5-3dc9-44fe-9ef5-d6e5d7ce469a admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:17.826427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5beb5-3dc9-44fe-9ef5-d6e5d7ce469a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:17.828309 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5beb5-3dc9-44fe-9ef5-d6e5d7ce469a admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:17.855724 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1194/2380] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:20:17 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:20:17.859823 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112af1fa-ab4d-4d06-9a06-d7dfc32f3da9 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:17.870296 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112af1fa-ab4d-4d06-9a06-d7dfc32f3da9 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:17.928942 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493cf028-f90c-4a98-9251-466244edf2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:17.929539 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493cf028-f90c-4a98-9251-466244edf2a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:17.929822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493cf028-f90c-4a98-9251-466244edf2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:17.930099 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493cf028-f90c-4a98-9251-466244edf2a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:17.930726 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493cf028-f90c-4a98-9251-466244edf2a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:17.932626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493cf028-f90c-4a98-9251-466244edf2a1 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:17.959055 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1187/2381] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:20:17 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:20:17.960331 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5beb5-3dc9-44fe-9ef5-d6e5d7ce469a admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:17.971198 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5beb5-3dc9-44fe-9ef5-d6e5d7ce469a admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:17.975823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8dfa47-48eb-4dfd-882f-3a1f7a3f163d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:17.976417 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8dfa47-48eb-4dfd-882f-3a1f7a3f163d admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:17.976729 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8dfa47-48eb-4dfd-882f-3a1f7a3f163d admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:17.977004 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8dfa47-48eb-4dfd-882f-3a1f7a3f163d admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:17.977640 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8dfa47-48eb-4dfd-882f-3a1f7a3f163d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:17.979221 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8dfa47-48eb-4dfd-882f-3a1f7a3f163d admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:17.989986 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1195/2382] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:20:17.993986 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493cf028-f90c-4a98-9251-466244edf2a1 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:18.004397 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493cf028-f90c-4a98-9251-466244edf2a1 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:18.009070 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11018120-107b-434a-8507-7a228dbb8f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:18.009653 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11018120-107b-434a-8507-7a228dbb8f88 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:18.009932 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11018120-107b-434a-8507-7a228dbb8f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:18.010206 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11018120-107b-434a-8507-7a228dbb8f88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:18.010831 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11018120-107b-434a-8507-7a228dbb8f88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:18.012392 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11018120-107b-434a-8507-7a228dbb8f88 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:18.022027 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1188/2383] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:20:18.026034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8dfa47-48eb-4dfd-882f-3a1f7a3f163d admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:18.036822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8dfa47-48eb-4dfd-882f-3a1f7a3f163d admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:18.041579 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f471ee7-0c65-4c48-9b36-ca8d524e0119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:18.042233 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f471ee7-0c65-4c48-9b36-ca8d524e0119 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:18.042515 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f471ee7-0c65-4c48-9b36-ca8d524e0119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:18.042791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f471ee7-0c65-4c48-9b36-ca8d524e0119 admin admin] PATH_INFO: `/v3/projects/b5ceff4f6c274c01b19ad8415564ec01/users/68930b49538a4b4288d91a0b37fe0277/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:18.058642 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f471ee7-0c65-4c48-9b36-ca8d524e0119 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ceff4f6c274c01b19ad8415564ec01, user_id=68930b49538a4b4288d91a0b37fe0277, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:18.061958 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f471ee7-0c65-4c48-9b36-ca8d524e0119 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:18.077010 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1196/2384] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:20:18 2022] PUT /identity/v3/projects/b5ceff4f6c274c01b19ad8415564ec01/users/68930b49538a4b4288d91a0b37fe0277/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:18.087388 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8f5d3-cf7c-40f0-ae82-5daa4a95de89 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:18.087388 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8f5d3-cf7c-40f0-ae82-5daa4a95de89 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:18.087609 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b8f5d3-cf7c-40f0-ae82-5daa4a95de89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:18.125288 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-14b8f5d3-cf7c-40f0-ae82-5daa4a95de89 None None] MFA Rules not processed for user `68930b49538a4b4288d91a0b37fe0277`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:18.126514 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-14b8f5d3-cf7c-40f0-ae82-5daa4a95de89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:18.193434 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1189/2385] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:18.204342 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f471ee7-0c65-4c48-9b36-ca8d524e0119 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:18.206113 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-1f471ee7-0c65-4c48-9b36-ca8d524e0119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:18.267919 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f471ee7-0c65-4c48-9b36-ca8d524e0119 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:18.271731 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acee513-ebe5-4187-9577-6da254ad27f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:18.272382 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acee513-ebe5-4187-9577-6da254ad27f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:18.273893 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acee513-ebe5-4187-9577-6da254ad27f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:18.273893 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acee513-ebe5-4187-9577-6da254ad27f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:18.282891 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acee513-ebe5-4187-9577-6da254ad27f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:18.284318 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acee513-ebe5-4187-9577-6da254ad27f3 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:18.293590 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1197/2386] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:20:28.179312 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8f5d3-cf7c-40f0-ae82-5daa4a95de89 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:28.228847 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:28.254919 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b8f5d3-cf7c-40f0-ae82-5daa4a95de89 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:28.260072 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8d2f2-1324-45a7-a34f-67b28abc3160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:28.260785 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8d2f2-1324-45a7-a34f-67b28abc3160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:28.261070 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8d2f2-1324-45a7-a34f-67b28abc3160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:28.261348 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8d2f2-1324-45a7-a34f-67b28abc3160 admin admin] PATH_INFO: `/v3/users/68930b49538a4b4288d91a0b37fe0277` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:28.262489 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8d2f2-1324-45a7-a34f-67b28abc3160 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68930b49538a4b4288d91a0b37fe0277)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:28.264821 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8d2f2-1324-45a7-a34f-67b28abc3160 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:28.289409 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:28.315898 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f0d8d2f2-1324-45a7-a34f-67b28abc3160 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68930b49538a4b4288d91a0b37fe0277'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:28.322780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f0d8d2f2-1324-45a7-a34f-67b28abc3160 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68930b49538a4b4288d91a0b37fe0277'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:28.335209 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f0d8d2f2-1324-45a7-a34f-67b28abc3160 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68930b49538a4b4288d91a0b37fe0277'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:28.335912 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f0d8d2f2-1324-45a7-a34f-67b28abc3160 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68930b49538a4b4288d91a0b37fe0277'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:28.336545 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-f0d8d2f2-1324-45a7-a34f-67b28abc3160 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68930b49538a4b4288d91a0b37fe0277'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:28.351710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1190/2387] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:20:28 2022] DELETE /identity/v3/users/68930b49538a4b4288d91a0b37fe0277 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:28.356199 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f45f4-eb02-4805-a79c-1045aec0e0f5 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:28.356534 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f45f4-eb02-4805-a79c-1045aec0e0f5 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:28.356840 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f45f4-eb02-4805-a79c-1045aec0e0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:28.377983 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-be9f45f4-eb02-4805-a79c-1045aec0e0f5 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:28.379231 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-be9f45f4-eb02-4805-a79c-1045aec0e0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:28.444716 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1198/2388] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:20:28.454098 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8d2f2-1324-45a7-a34f-67b28abc3160 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:28.455520 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f0d8d2f2-1324-45a7-a34f-67b28abc3160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:28.519755 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8d2f2-1324-45a7-a34f-67b28abc3160 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:28.523615 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb19ea-b719-4b81-b158-827737ad18c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:28.524298 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb19ea-b719-4b81-b158-827737ad18c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:28.524686 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb19ea-b719-4b81-b158-827737ad18c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:28.524963 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb19ea-b719-4b81-b158-827737ad18c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:28.534517 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb19ea-b719-4b81-b158-827737ad18c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:28.535921 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb19ea-b719-4b81-b158-827737ad18c5 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:28.545817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1191/2389] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:20:28.730200 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f45f4-eb02-4805-a79c-1045aec0e0f5 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:28.731830 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-be9f45f4-eb02-4805-a79c-1045aec0e0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:28.746313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f45f4-eb02-4805-a79c-1045aec0e0f5 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:28.752065 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0644aca-aec3-4a87-81f7-b99f97ae8873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:28.752743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0644aca-aec3-4a87-81f7-b99f97ae8873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:28.753030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0644aca-aec3-4a87-81f7-b99f97ae8873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:28.753310 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0644aca-aec3-4a87-81f7-b99f97ae8873 admin admin] PATH_INFO: `/v3/users/d17ddde063034ef9935c0dd5b55666b7` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:28.754460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0644aca-aec3-4a87-81f7-b99f97ae8873 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d17ddde063034ef9935c0dd5b55666b7)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:28.756397 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0644aca-aec3-4a87-81f7-b99f97ae8873 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:28.805488 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-f0644aca-aec3-4a87-81f7-b99f97ae8873 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd17ddde063034ef9935c0dd5b55666b7'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:28.806248 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-f0644aca-aec3-4a87-81f7-b99f97ae8873 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd17ddde063034ef9935c0dd5b55666b7'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:28.815588 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-f0644aca-aec3-4a87-81f7-b99f97ae8873 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd17ddde063034ef9935c0dd5b55666b7'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:28.826400 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb19ea-b719-4b81-b158-827737ad18c5 service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:28.828123 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-cffb19ea-b719-4b81-b158-827737ad18c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:28.829616 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-f0644aca-aec3-4a87-81f7-b99f97ae8873 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd17ddde063034ef9935c0dd5b55666b7'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:28.837086 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-f0644aca-aec3-4a87-81f7-b99f97ae8873 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd17ddde063034ef9935c0dd5b55666b7'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:28.849273 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb19ea-b719-4b81-b158-827737ad18c5 service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:28.853760 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1199/2390] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:20:28 2022] DELETE /identity/v3/users/d17ddde063034ef9935c0dd5b55666b7 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:28.858186 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d1323-50c9-4a74-9174-25ffa1307862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:28.858896 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d1323-50c9-4a74-9174-25ffa1307862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:28.859191 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d1323-50c9-4a74-9174-25ffa1307862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:28.859470 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d1323-50c9-4a74-9174-25ffa1307862 admin admin] PATH_INFO: `/v3/projects/b5ceff4f6c274c01b19ad8415564ec01` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:28.860832 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584d1323-50c9-4a74-9174-25ffa1307862 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5ceff4f6c274c01b19ad8415564ec01)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:28.862952 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584d1323-50c9-4a74-9174-25ffa1307862 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:28.916773 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-584d1323-50c9-4a74-9174-25ffa1307862 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5ceff4f6c274c01b19ad8415564ec01', 'request_id': 'req-584d1323-50c9-4a74-9174-25ffa1307862'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:28.924572 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-584d1323-50c9-4a74-9174-25ffa1307862 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5ceff4f6c274c01b19ad8415564ec01', 'request_id': 'req-584d1323-50c9-4a74-9174-25ffa1307862'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:28.927350 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:28.932879 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-584d1323-50c9-4a74-9174-25ffa1307862 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5ceff4f6c274c01b19ad8415564ec01', 'request_id': 'req-584d1323-50c9-4a74-9174-25ffa1307862'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:28.945003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-584d1323-50c9-4a74-9174-25ffa1307862 admin admin] The token cache is being invalidate because project b5ceff4f6c274c01b19ad8415564ec01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:20:28.945379 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-584d1323-50c9-4a74-9174-25ffa1307862 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:28.946081 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-584d1323-50c9-4a74-9174-25ffa1307862 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:28.947658 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1192/2391] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:20:28 2022] DELETE /identity/v3/projects/b5ceff4f6c274c01b19ad8415564ec01 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:29.046887 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0644aca-aec3-4a87-81f7-b99f97ae8873 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:29.048555 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f0644aca-aec3-4a87-81f7-b99f97ae8873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:29.132889 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0644aca-aec3-4a87-81f7-b99f97ae8873 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:29.152575 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:29.199772 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07b1af-651d-4613-a664-3acbc1d67e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:29.200412 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07b1af-651d-4613-a664-3acbc1d67e19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:29.201237 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07b1af-651d-4613-a664-3acbc1d67e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:29.201589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07b1af-651d-4613-a664-3acbc1d67e19 admin admin] PATH_INFO: `/v3/projects/5dc246ad81ed43a0b7170adc05e047f4` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:29.202942 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe07b1af-651d-4613-a664-3acbc1d67e19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dc246ad81ed43a0b7170adc05e047f4)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:29.204976 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe07b1af-651d-4613-a664-3acbc1d67e19 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:29.251119 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fe07b1af-651d-4613-a664-3acbc1d67e19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dc246ad81ed43a0b7170adc05e047f4', 'request_id': 'req-fe07b1af-651d-4613-a664-3acbc1d67e19'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:29.258486 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fe07b1af-651d-4613-a664-3acbc1d67e19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dc246ad81ed43a0b7170adc05e047f4', 'request_id': 'req-fe07b1af-651d-4613-a664-3acbc1d67e19'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:29.264749 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fe07b1af-651d-4613-a664-3acbc1d67e19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dc246ad81ed43a0b7170adc05e047f4', 'request_id': 'req-fe07b1af-651d-4613-a664-3acbc1d67e19'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:29.275684 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fe07b1af-651d-4613-a664-3acbc1d67e19 admin admin] The token cache is being invalidate because project 5dc246ad81ed43a0b7170adc05e047f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:20:29.276080 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fe07b1af-651d-4613-a664-3acbc1d67e19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:29.276758 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-fe07b1af-651d-4613-a664-3acbc1d67e19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:29.278282 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1200/2392] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:20:29 2022] DELETE /identity/v3/projects/5dc246ad81ed43a0b7170adc05e047f4 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:20:29.282316 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d1323-50c9-4a74-9174-25ffa1307862 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:29.283700 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-584d1323-50c9-4a74-9174-25ffa1307862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:29.296343 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d1323-50c9-4a74-9174-25ffa1307862 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:29.352254 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897fb3e-8bbb-48e5-b4ea-2486fa3f7ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:29.352919 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897fb3e-8bbb-48e5-b4ea-2486fa3f7ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:29.353206 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897fb3e-8bbb-48e5-b4ea-2486fa3f7ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:29.353483 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897fb3e-8bbb-48e5-b4ea-2486fa3f7ff8 admin admin] PATH_INFO: `/v3/users/8cd15bbfa7e64810a6cef86ae58acd8c` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:29.354606 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897fb3e-8bbb-48e5-b4ea-2486fa3f7ff8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cd15bbfa7e64810a6cef86ae58acd8c)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:29.356512 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897fb3e-8bbb-48e5-b4ea-2486fa3f7ff8 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:29.403161 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-2897fb3e-8bbb-48e5-b4ea-2486fa3f7ff8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cd15bbfa7e64810a6cef86ae58acd8c'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:29.410353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-2897fb3e-8bbb-48e5-b4ea-2486fa3f7ff8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cd15bbfa7e64810a6cef86ae58acd8c'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:29.422379 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-2897fb3e-8bbb-48e5-b4ea-2486fa3f7ff8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cd15bbfa7e64810a6cef86ae58acd8c'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:29.423034 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-2897fb3e-8bbb-48e5-b4ea-2486fa3f7ff8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cd15bbfa7e64810a6cef86ae58acd8c'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:29.423647 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-2897fb3e-8bbb-48e5-b4ea-2486fa3f7ff8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cd15bbfa7e64810a6cef86ae58acd8c'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:29.436138 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1193/2393] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:20:29 2022] DELETE /identity/v3/users/8cd15bbfa7e64810a6cef86ae58acd8c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:29.615132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07b1af-651d-4613-a664-3acbc1d67e19 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:29.616645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-fe07b1af-651d-4613-a664-3acbc1d67e19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:29.630607 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07b1af-651d-4613-a664-3acbc1d67e19 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:29.688209 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5907b47f-aa2c-4180-ad91-03dbde839984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:29.688810 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907b47f-aa2c-4180-ad91-03dbde839984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:29.689098 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907b47f-aa2c-4180-ad91-03dbde839984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:29.689379 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907b47f-aa2c-4180-ad91-03dbde839984 admin admin] PATH_INFO: `/v3/projects/104e33ddffdc49af9baa6ecda4d054d5` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:29.690498 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5907b47f-aa2c-4180-ad91-03dbde839984 admin admin] RBAC: Authorizing `identity:delete_project(project_id=104e33ddffdc49af9baa6ecda4d054d5)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:29.692390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5907b47f-aa2c-4180-ad91-03dbde839984 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:29.738602 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5907b47f-aa2c-4180-ad91-03dbde839984 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '104e33ddffdc49af9baa6ecda4d054d5', 'request_id': 'req-5907b47f-aa2c-4180-ad91-03dbde839984'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:29.741386 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:29.746848 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5907b47f-aa2c-4180-ad91-03dbde839984 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '104e33ddffdc49af9baa6ecda4d054d5', 'request_id': 'req-5907b47f-aa2c-4180-ad91-03dbde839984'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:29.754654 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5907b47f-aa2c-4180-ad91-03dbde839984 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '104e33ddffdc49af9baa6ecda4d054d5', 'request_id': 'req-5907b47f-aa2c-4180-ad91-03dbde839984'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:29.765867 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5907b47f-aa2c-4180-ad91-03dbde839984 admin admin] The token cache is being invalidate because project 104e33ddffdc49af9baa6ecda4d054d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:20:29.766170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5907b47f-aa2c-4180-ad91-03dbde839984 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:29.766779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5907b47f-aa2c-4180-ad91-03dbde839984 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:29.768246 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1201/2394] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:20:29 2022] DELETE /identity/v3/projects/104e33ddffdc49af9baa6ecda4d054d5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:29.773248 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb979b44-2f25-4e1d-8b23-0cee66aa40d8 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:29.773543 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb979b44-2f25-4e1d-8b23-0cee66aa40d8 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:29.773826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb979b44-2f25-4e1d-8b23-0cee66aa40d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:29.793140 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-bb979b44-2f25-4e1d-8b23-0cee66aa40d8 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:29.794353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-bb979b44-2f25-4e1d-8b23-0cee66aa40d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:29.853351 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1194/2395] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:29.862983 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad51f8-36d9-4d68-9a0a-f770decb12f3 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:29.863270 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad51f8-36d9-4d68-9a0a-f770decb12f3 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:29.863545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad51f8-36d9-4d68-9a0a-f770decb12f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:29.883889 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-69ad51f8-36d9-4d68-9a0a-f770decb12f3 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:29.885126 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-69ad51f8-36d9-4d68-9a0a-f770decb12f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:29.893270 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1202/2396] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:29.897485 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb979b44-2f25-4e1d-8b23-0cee66aa40d8 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:29.903645 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:29.907630 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb979b44-2f25-4e1d-8b23-0cee66aa40d8 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:29.913720 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d9c40-7f10-457c-a6ca-80e969acc399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:29.914509 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d9c40-7f10-457c-a6ca-80e969acc399 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:29.915006 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d9c40-7f10-457c-a6ca-80e969acc399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:29.915461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d9c40-7f10-457c-a6ca-80e969acc399 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:29.916141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d9c40-7f10-457c-a6ca-80e969acc399 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:20:29.916480 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d9c40-7f10-457c-a6ca-80e969acc399 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:29.918087 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d9c40-7f10-457c-a6ca-80e969acc399 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:29.930302 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1195/2397] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:20:29.934298 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad51f8-36d9-4d68-9a0a-f770decb12f3 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:29.944158 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad51f8-36d9-4d68-9a0a-f770decb12f3 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:29.948697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7313b-6720-47b0-9d8f-9f8634ec3cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:29.949286 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7313b-6720-47b0-9d8f-9f8634ec3cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:29.949565 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7313b-6720-47b0-9d8f-9f8634ec3cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:29.949838 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7313b-6720-47b0-9d8f-9f8634ec3cc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:29.950489 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7313b-6720-47b0-9d8f-9f8634ec3cc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:29.952510 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7313b-6720-47b0-9d8f-9f8634ec3cc9 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:29.978823 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1203/2398] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:20:29 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:20:29.983452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d9c40-7f10-457c-a6ca-80e969acc399 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:29.993379 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d9c40-7f10-457c-a6ca-80e969acc399 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:30.051931 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9413c82-79ae-4de4-b062-b24857095514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:30.052562 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9413c82-79ae-4de4-b062-b24857095514 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:30.052852 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9413c82-79ae-4de4-b062-b24857095514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:30.053133 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9413c82-79ae-4de4-b062-b24857095514 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:30.053759 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9413c82-79ae-4de4-b062-b24857095514 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:30.055662 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9413c82-79ae-4de4-b062-b24857095514 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:30.077098 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1196/2399] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:20:29 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:20:30.081228 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7313b-6720-47b0-9d8f-9f8634ec3cc9 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:30.091839 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7313b-6720-47b0-9d8f-9f8634ec3cc9 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:30.096858 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5ce22-b478-4d78-a7f8-f8803e7da863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:30.097447 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5ce22-b478-4d78-a7f8-f8803e7da863 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:30.097726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5ce22-b478-4d78-a7f8-f8803e7da863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:30.098000 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5ce22-b478-4d78-a7f8-f8803e7da863 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:30.098630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5ce22-b478-4d78-a7f8-f8803e7da863 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:30.100396 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5ce22-b478-4d78-a7f8-f8803e7da863 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:30.111100 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1204/2400] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:20:30.115464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9413c82-79ae-4de4-b062-b24857095514 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:30.126381 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9413c82-79ae-4de4-b062-b24857095514 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:30.131167 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571c2a6-1258-4092-bd9a-73a2e74468c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:30.131822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571c2a6-1258-4092-bd9a-73a2e74468c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:30.132108 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571c2a6-1258-4092-bd9a-73a2e74468c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:30.132387 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571c2a6-1258-4092-bd9a-73a2e74468c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:30.133070 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571c2a6-1258-4092-bd9a-73a2e74468c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:30.134671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571c2a6-1258-4092-bd9a-73a2e74468c2 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:30.146994 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1197/2401] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:20:30.151169 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5ce22-b478-4d78-a7f8-f8803e7da863 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:30.161377 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5ce22-b478-4d78-a7f8-f8803e7da863 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:30.166665 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9a0963-31d5-468e-9f2e-26ac5710ed24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:30.167314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a0963-31d5-468e-9f2e-26ac5710ed24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:30.167596 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a0963-31d5-468e-9f2e-26ac5710ed24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:30.167914 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a0963-31d5-468e-9f2e-26ac5710ed24 admin admin] PATH_INFO: `/v3/projects/4f1e93b52a1d4d4fa15b29fd36458f81/users/7773d4b277e040519b2920b78ff98d68/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:30.182344 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9a0963-31d5-468e-9f2e-26ac5710ed24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f1e93b52a1d4d4fa15b29fd36458f81, user_id=7773d4b277e040519b2920b78ff98d68, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:30.185774 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9a0963-31d5-468e-9f2e-26ac5710ed24 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:30.199389 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1205/2402] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:20:30 2022] PUT /identity/v3/projects/4f1e93b52a1d4d4fa15b29fd36458f81/users/7773d4b277e040519b2920b78ff98d68/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:30.211104 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71abea-9372-40fc-93a1-5e6f5934ad80 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:30.211398 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71abea-9372-40fc-93a1-5e6f5934ad80 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:30.211677 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71abea-9372-40fc-93a1-5e6f5934ad80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:30.239867 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-5d71abea-9372-40fc-93a1-5e6f5934ad80 None None] MFA Rules not processed for user `7773d4b277e040519b2920b78ff98d68`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:30.241106 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5d71abea-9372-40fc-93a1-5e6f5934ad80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:30.304658 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1198/2403] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:20:30.318499 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9a0963-31d5-468e-9f2e-26ac5710ed24 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:30.320182 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-de9a0963-31d5-468e-9f2e-26ac5710ed24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:30.377477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9a0963-31d5-468e-9f2e-26ac5710ed24 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:30.381061 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbdcdda-4e4b-49a4-b0ad-7bdb9114aa08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:30.381659 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbdcdda-4e4b-49a4-b0ad-7bdb9114aa08 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:30.381941 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbdcdda-4e4b-49a4-b0ad-7bdb9114aa08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:30.382249 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbdcdda-4e4b-49a4-b0ad-7bdb9114aa08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:30.390695 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbdcdda-4e4b-49a4-b0ad-7bdb9114aa08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:30.392158 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbdcdda-4e4b-49a4-b0ad-7bdb9114aa08 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:30.400942 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1206/2404] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:20:30.420179 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71abea-9372-40fc-93a1-5e6f5934ad80 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:30.481648 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71abea-9372-40fc-93a1-5e6f5934ad80 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:30.486714 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e6b57-419a-427a-9d4c-464f69234754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:30.487338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e6b57-419a-427a-9d4c-464f69234754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:30.487647 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e6b57-419a-427a-9d4c-464f69234754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:30.487929 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e6b57-419a-427a-9d4c-464f69234754 admin admin] PATH_INFO: `/v3/users/7773d4b277e040519b2920b78ff98d68` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:30.489083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751e6b57-419a-427a-9d4c-464f69234754 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7773d4b277e040519b2920b78ff98d68)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:30.491244 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751e6b57-419a-427a-9d4c-464f69234754 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:30.492759 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:30.535864 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-751e6b57-419a-427a-9d4c-464f69234754 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7773d4b277e040519b2920b78ff98d68'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:30.542836 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-751e6b57-419a-427a-9d4c-464f69234754 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7773d4b277e040519b2920b78ff98d68'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:30.555546 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-751e6b57-419a-427a-9d4c-464f69234754 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7773d4b277e040519b2920b78ff98d68'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:30.556223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-751e6b57-419a-427a-9d4c-464f69234754 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7773d4b277e040519b2920b78ff98d68'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:30.556867 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-751e6b57-419a-427a-9d4c-464f69234754 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7773d4b277e040519b2920b78ff98d68'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:30.570052 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1199/2405] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:20:30 2022] DELETE /identity/v3/users/7773d4b277e040519b2920b78ff98d68 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:30.574673 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13b1c2-aec7-451b-8aab-42a01c107eb9 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:30.574968 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13b1c2-aec7-451b-8aab-42a01c107eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:30.575250 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13b1c2-aec7-451b-8aab-42a01c107eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:30.594951 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-1d13b1c2-aec7-451b-8aab-42a01c107eb9 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:30.596134 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-1d13b1c2-aec7-451b-8aab-42a01c107eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:30.655470 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1207/2406] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:30.665182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e6b57-419a-427a-9d4c-464f69234754 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:30.666574 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-751e6b57-419a-427a-9d4c-464f69234754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:30.724141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e6b57-419a-427a-9d4c-464f69234754 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:30.727756 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c030eb-b00d-4f68-9de6-a9d8e88f2ff5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:30.728352 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c030eb-b00d-4f68-9de6-a9d8e88f2ff5 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:30.728661 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c030eb-b00d-4f68-9de6-a9d8e88f2ff5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:30.728933 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c030eb-b00d-4f68-9de6-a9d8e88f2ff5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:30.738525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c030eb-b00d-4f68-9de6-a9d8e88f2ff5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:30.740136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c030eb-b00d-4f68-9de6-a9d8e88f2ff5 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:30.749478 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1200/2407] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:20:31.015676 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13b1c2-aec7-451b-8aab-42a01c107eb9 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:31.017155 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-1d13b1c2-aec7-451b-8aab-42a01c107eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:31.032073 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13b1c2-aec7-451b-8aab-42a01c107eb9 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:31.036744 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c830db-4d9b-4b86-802e-8ffb1fa55fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:31.037383 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c830db-4d9b-4b86-802e-8ffb1fa55fd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:31.037660 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c830db-4d9b-4b86-802e-8ffb1fa55fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:31.037944 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c830db-4d9b-4b86-802e-8ffb1fa55fd6 admin admin] PATH_INFO: `/v3/projects/4f1e93b52a1d4d4fa15b29fd36458f81` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:31.039066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c830db-4d9b-4b86-802e-8ffb1fa55fd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f1e93b52a1d4d4fa15b29fd36458f81)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:31.040987 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c830db-4d9b-4b86-802e-8ffb1fa55fd6 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:31.057976 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:31.089701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-19c830db-4d9b-4b86-802e-8ffb1fa55fd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f1e93b52a1d4d4fa15b29fd36458f81', 'request_id': 'req-19c830db-4d9b-4b86-802e-8ffb1fa55fd6'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:31.097477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-19c830db-4d9b-4b86-802e-8ffb1fa55fd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f1e93b52a1d4d4fa15b29fd36458f81', 'request_id': 'req-19c830db-4d9b-4b86-802e-8ffb1fa55fd6'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:31.104520 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-19c830db-4d9b-4b86-802e-8ffb1fa55fd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f1e93b52a1d4d4fa15b29fd36458f81', 'request_id': 'req-19c830db-4d9b-4b86-802e-8ffb1fa55fd6'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:31.116713 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-19c830db-4d9b-4b86-802e-8ffb1fa55fd6 admin admin] The token cache is being invalidate because project 4f1e93b52a1d4d4fa15b29fd36458f81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:20:31.117011 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-19c830db-4d9b-4b86-802e-8ffb1fa55fd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:31.117669 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-19c830db-4d9b-4b86-802e-8ffb1fa55fd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:31.119158 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1208/2408] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:20:31 2022] DELETE /identity/v3/projects/4f1e93b52a1d4d4fa15b29fd36458f81 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:31.124367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dae787-0a0d-4d12-b062-89c87c813f3d None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:31.124684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dae787-0a0d-4d12-b062-89c87c813f3d None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:31.124959 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dae787-0a0d-4d12-b062-89c87c813f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:31.145865 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-b9dae787-0a0d-4d12-b062-89c87c813f3d None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:31.147108 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b9dae787-0a0d-4d12-b062-89c87c813f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:31.205898 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1201/2409] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:31.213819 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960a2ca-8aac-4797-8487-a54378f213f4 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:31.214112 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960a2ca-8aac-4797-8487-a54378f213f4 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:31.214425 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960a2ca-8aac-4797-8487-a54378f213f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:31.234162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-a960a2ca-8aac-4797-8487-a54378f213f4 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:31.235347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a960a2ca-8aac-4797-8487-a54378f213f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:31.243390 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1209/2410] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:31.247503 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dae787-0a0d-4d12-b062-89c87c813f3d None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:31.257765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dae787-0a0d-4d12-b062-89c87c813f3d None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:31.262353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb7967-1458-45ab-96b6-724d6245c1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:31.262887 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb7967-1458-45ab-96b6-724d6245c1b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:31.263169 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb7967-1458-45ab-96b6-724d6245c1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:31.263510 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb7967-1458-45ab-96b6-724d6245c1b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:31.264126 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb7967-1458-45ab-96b6-724d6245c1b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:20:31.264422 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb7967-1458-45ab-96b6-724d6245c1b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:31.266004 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb7967-1458-45ab-96b6-724d6245c1b5 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:31.277236 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1202/2411] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:20:31.281553 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960a2ca-8aac-4797-8487-a54378f213f4 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:31.290517 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960a2ca-8aac-4797-8487-a54378f213f4 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:31.295718 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5079e-fef9-4e00-894f-f1661ba4173c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:31.296302 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5079e-fef9-4e00-894f-f1661ba4173c admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:31.296613 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5079e-fef9-4e00-894f-f1661ba4173c admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:31.296890 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5079e-fef9-4e00-894f-f1661ba4173c admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:31.297522 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f5079e-fef9-4e00-894f-f1661ba4173c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:31.299410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f5079e-fef9-4e00-894f-f1661ba4173c admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:31.327830 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1210/2412] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:20:31 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:20:31.331900 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb7967-1458-45ab-96b6-724d6245c1b5 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:31.341699 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb7967-1458-45ab-96b6-724d6245c1b5 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:31.346252 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:31.399291 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d090611-a8f4-4e1a-b7da-7cd506223ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:31.399893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d090611-a8f4-4e1a-b7da-7cd506223ddb admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:31.400175 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d090611-a8f4-4e1a-b7da-7cd506223ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:31.400448 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d090611-a8f4-4e1a-b7da-7cd506223ddb admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:31.401095 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d090611-a8f4-4e1a-b7da-7cd506223ddb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:31.402965 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d090611-a8f4-4e1a-b7da-7cd506223ddb admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:31.424258 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1203/2413] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:20:31 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:20:31.428432 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5079e-fef9-4e00-894f-f1661ba4173c admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:31.439190 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5079e-fef9-4e00-894f-f1661ba4173c admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:31.443798 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81e987-9f26-4f44-8cdc-574b46048ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:31.444389 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81e987-9f26-4f44-8cdc-574b46048ade admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:31.444704 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81e987-9f26-4f44-8cdc-574b46048ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:31.444982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81e987-9f26-4f44-8cdc-574b46048ade admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:31.445637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81e987-9f26-4f44-8cdc-574b46048ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:31.447189 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81e987-9f26-4f44-8cdc-574b46048ade admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:31.456955 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1211/2414] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:20:31.460715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d090611-a8f4-4e1a-b7da-7cd506223ddb admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:31.471471 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d090611-a8f4-4e1a-b7da-7cd506223ddb admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:31.476106 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad68ec0a-a460-40fc-ab3d-b018e94277e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:31.476718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68ec0a-a460-40fc-ab3d-b018e94277e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:31.476999 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68ec0a-a460-40fc-ab3d-b018e94277e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:31.477272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68ec0a-a460-40fc-ab3d-b018e94277e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:31.478912 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad68ec0a-a460-40fc-ab3d-b018e94277e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:31.480469 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad68ec0a-a460-40fc-ab3d-b018e94277e4 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:31.492920 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1204/2415] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:20:31.497078 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81e987-9f26-4f44-8cdc-574b46048ade admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:31.506928 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81e987-9f26-4f44-8cdc-574b46048ade admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:31.511733 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa22cf-f92c-498a-9ed7-126330815e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:31.512420 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa22cf-f92c-498a-9ed7-126330815e66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:31.512690 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa22cf-f92c-498a-9ed7-126330815e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:31.512972 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa22cf-f92c-498a-9ed7-126330815e66 admin admin] PATH_INFO: `/v3/projects/0f7b4d0df89d49c994bf205a79d0e03a/users/be90a53cc55742e080a422c195dfee99/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:31.527436 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa22cf-f92c-498a-9ed7-126330815e66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f7b4d0df89d49c994bf205a79d0e03a, user_id=be90a53cc55742e080a422c195dfee99, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:31.530773 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa22cf-f92c-498a-9ed7-126330815e66 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:31.542976 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1212/2416] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:20:31 2022] PUT /identity/v3/projects/0f7b4d0df89d49c994bf205a79d0e03a/users/be90a53cc55742e080a422c195dfee99/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:20:31.611518 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ad3c7-db2d-46e7-804a-3b7afea1043e None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:31.611804 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ad3c7-db2d-46e7-804a-3b7afea1043e None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:31.612078 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ad3c7-db2d-46e7-804a-3b7afea1043e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:31.640325 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-145ad3c7-db2d-46e7-804a-3b7afea1043e None None] MFA Rules not processed for user `be90a53cc55742e080a422c195dfee99`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:31.641559 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-145ad3c7-db2d-46e7-804a-3b7afea1043e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:31.705504 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1205/2417] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:31.717219 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa22cf-f92c-498a-9ed7-126330815e66 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:31.719391 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d5aa22cf-f92c-498a-9ed7-126330815e66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:31.775272 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa22cf-f92c-498a-9ed7-126330815e66 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:31.779316 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b3aa9-fed0-4d4a-8dd8-fd2b8b90865d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:31.780010 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b3aa9-fed0-4d4a-8dd8-fd2b8b90865d service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:31.780291 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b3aa9-fed0-4d4a-8dd8-fd2b8b90865d service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:31.780591 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b3aa9-fed0-4d4a-8dd8-fd2b8b90865d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:31.788894 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b3aa9-fed0-4d4a-8dd8-fd2b8b90865d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:31.790302 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6b3aa9-fed0-4d4a-8dd8-fd2b8b90865d service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:31.799695 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1213/2418] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:20:31.821357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ad3c7-db2d-46e7-804a-3b7afea1043e None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:31.887752 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ad3c7-db2d-46e7-804a-3b7afea1043e None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:31.892947 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8cae7-ef68-4145-b84a-bbe00930c8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:31.893575 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8cae7-ef68-4145-b84a-bbe00930c8ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:31.893861 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8cae7-ef68-4145-b84a-bbe00930c8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:31.894141 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8cae7-ef68-4145-b84a-bbe00930c8ed admin admin] PATH_INFO: `/v3/users/be90a53cc55742e080a422c195dfee99` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:31.895268 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8cae7-ef68-4145-b84a-bbe00930c8ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=be90a53cc55742e080a422c195dfee99)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:31.897227 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8cae7-ef68-4145-b84a-bbe00930c8ed admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:31.918556 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:31.943965 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-6dd8cae7-ef68-4145-b84a-bbe00930c8ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be90a53cc55742e080a422c195dfee99'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:31.951882 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-6dd8cae7-ef68-4145-b84a-bbe00930c8ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be90a53cc55742e080a422c195dfee99'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:31.963691 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-6dd8cae7-ef68-4145-b84a-bbe00930c8ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be90a53cc55742e080a422c195dfee99'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:31.964365 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-6dd8cae7-ef68-4145-b84a-bbe00930c8ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be90a53cc55742e080a422c195dfee99'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:31.964999 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-6dd8cae7-ef68-4145-b84a-bbe00930c8ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be90a53cc55742e080a422c195dfee99'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:31.978815 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1206/2419] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:20:31 2022] DELETE /identity/v3/users/be90a53cc55742e080a422c195dfee99 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:31.983603 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74a471-226a-4b53-a1c7-26d117e91ee8 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:31.983889 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74a471-226a-4b53-a1c7-26d117e91ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:31.984168 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74a471-226a-4b53-a1c7-26d117e91ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:32.003311 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-df74a471-226a-4b53-a1c7-26d117e91ee8 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:32.004515 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-df74a471-226a-4b53-a1c7-26d117e91ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:32.041814 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:32.062968 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1214/2420] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:32.072603 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8cae7-ef68-4145-b84a-bbe00930c8ed admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:32.074010 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-6dd8cae7-ef68-4145-b84a-bbe00930c8ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:32.134099 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8cae7-ef68-4145-b84a-bbe00930c8ed admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:32.137698 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936320e-e048-4ea5-9b5c-f8da6377b9bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:32.138823 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936320e-e048-4ea5-9b5c-f8da6377b9bf service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:32.139263 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936320e-e048-4ea5-9b5c-f8da6377b9bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:32.139540 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936320e-e048-4ea5-9b5c-f8da6377b9bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:32.148169 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936320e-e048-4ea5-9b5c-f8da6377b9bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:32.149601 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936320e-e048-4ea5-9b5c-f8da6377b9bf service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:32.159045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1207/2421] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:20:32.429551 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74a471-226a-4b53-a1c7-26d117e91ee8 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:32.431072 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-df74a471-226a-4b53-a1c7-26d117e91ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:32.443134 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74a471-226a-4b53-a1c7-26d117e91ee8 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:32.447771 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f6b32-14be-4667-bbc6-423852652c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:32.448405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f6b32-14be-4667-bbc6-423852652c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:32.448719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f6b32-14be-4667-bbc6-423852652c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:32.449010 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f6b32-14be-4667-bbc6-423852652c4e admin admin] PATH_INFO: `/v3/projects/0f7b4d0df89d49c994bf205a79d0e03a` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:32.450198 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f6b32-14be-4667-bbc6-423852652c4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f7b4d0df89d49c994bf205a79d0e03a)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:32.452131 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f6b32-14be-4667-bbc6-423852652c4e admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:32.499734 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-610f6b32-14be-4667-bbc6-423852652c4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f7b4d0df89d49c994bf205a79d0e03a', 'request_id': 'req-610f6b32-14be-4667-bbc6-423852652c4e'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:32.508007 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-610f6b32-14be-4667-bbc6-423852652c4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f7b4d0df89d49c994bf205a79d0e03a', 'request_id': 'req-610f6b32-14be-4667-bbc6-423852652c4e'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:32.513744 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-610f6b32-14be-4667-bbc6-423852652c4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f7b4d0df89d49c994bf205a79d0e03a', 'request_id': 'req-610f6b32-14be-4667-bbc6-423852652c4e'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:32.525009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-610f6b32-14be-4667-bbc6-423852652c4e admin admin] The token cache is being invalidate because project 0f7b4d0df89d49c994bf205a79d0e03a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:20:32.525302 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-610f6b32-14be-4667-bbc6-423852652c4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:32.525959 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-610f6b32-14be-4667-bbc6-423852652c4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:32.527446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1215/2422] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:20:32 2022] DELETE /identity/v3/projects/0f7b4d0df89d49c994bf205a79d0e03a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:32.533037 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827bb3d-bf54-419d-8a2b-5419463a5416 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:32.533525 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827bb3d-bf54-419d-8a2b-5419463a5416 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:32.533626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827bb3d-bf54-419d-8a2b-5419463a5416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:32.553040 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-4827bb3d-bf54-419d-8a2b-5419463a5416 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:32.554161 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-4827bb3d-bf54-419d-8a2b-5419463a5416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:32.612618 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1208/2423] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:32.620826 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243be3d1-3d1d-4af0-82c2-6979a4360f8d None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:32.621116 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243be3d1-3d1d-4af0-82c2-6979a4360f8d None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:32.621393 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243be3d1-3d1d-4af0-82c2-6979a4360f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:32.641809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-243be3d1-3d1d-4af0-82c2-6979a4360f8d None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:32.643003 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-243be3d1-3d1d-4af0-82c2-6979a4360f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:32.650729 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1216/2424] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:32.655508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827bb3d-bf54-419d-8a2b-5419463a5416 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:32.665096 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827bb3d-bf54-419d-8a2b-5419463a5416 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:32.669827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf78e3f-e5dc-404d-b970-d1d0dc5f8ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:32.670462 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf78e3f-e5dc-404d-b970-d1d0dc5f8ad6 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:32.670747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf78e3f-e5dc-404d-b970-d1d0dc5f8ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:32.671025 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf78e3f-e5dc-404d-b970-d1d0dc5f8ad6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:32.671675 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf78e3f-e5dc-404d-b970-d1d0dc5f8ad6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:20:32.671983 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf78e3f-e5dc-404d-b970-d1d0dc5f8ad6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:32.673584 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf78e3f-e5dc-404d-b970-d1d0dc5f8ad6 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:32.684991 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1209/2425] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:20:32.689206 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243be3d1-3d1d-4af0-82c2-6979a4360f8d None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:32.698414 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243be3d1-3d1d-4af0-82c2-6979a4360f8d None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:32.703955 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71388d9c-894b-42e6-82d3-cf134ef0ec37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:32.704593 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71388d9c-894b-42e6-82d3-cf134ef0ec37 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:32.704879 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71388d9c-894b-42e6-82d3-cf134ef0ec37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:32.705161 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71388d9c-894b-42e6-82d3-cf134ef0ec37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:32.705805 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71388d9c-894b-42e6-82d3-cf134ef0ec37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:32.707779 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71388d9c-894b-42e6-82d3-cf134ef0ec37 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:32.734962 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1217/2426] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:20:32 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:20:32.739116 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf78e3f-e5dc-404d-b970-d1d0dc5f8ad6 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:32.749844 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf78e3f-e5dc-404d-b970-d1d0dc5f8ad6 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:32.773569 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:32.808581 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98264e-1c7b-4320-b96a-141ee49caf2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:32.809119 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98264e-1c7b-4320-b96a-141ee49caf2e admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:32.809407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98264e-1c7b-4320-b96a-141ee49caf2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:32.809712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98264e-1c7b-4320-b96a-141ee49caf2e admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:32.811387 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c98264e-1c7b-4320-b96a-141ee49caf2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:32.813699 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c98264e-1c7b-4320-b96a-141ee49caf2e admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:32.834788 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1210/2427] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:20:32 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:20:32.838928 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71388d9c-894b-42e6-82d3-cf134ef0ec37 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:32.849318 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71388d9c-894b-42e6-82d3-cf134ef0ec37 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:32.853950 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bbbe16-1f45-45e6-9be8-0a33036dfb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:32.854544 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbbe16-1f45-45e6-9be8-0a33036dfb80 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:32.854826 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbbe16-1f45-45e6-9be8-0a33036dfb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:32.855101 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbbe16-1f45-45e6-9be8-0a33036dfb80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:32.855735 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bbbe16-1f45-45e6-9be8-0a33036dfb80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:32.857921 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bbbe16-1f45-45e6-9be8-0a33036dfb80 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:32.867836 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1218/2428] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:20:32.871544 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98264e-1c7b-4320-b96a-141ee49caf2e admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:32.886897 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98264e-1c7b-4320-b96a-141ee49caf2e admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:32.892156 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5bca69-6007-41d9-bc04-7674d5605eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:32.892792 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5bca69-6007-41d9-bc04-7674d5605eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:32.893080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5bca69-6007-41d9-bc04-7674d5605eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:32.893360 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5bca69-6007-41d9-bc04-7674d5605eb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:32.894000 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5bca69-6007-41d9-bc04-7674d5605eb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:32.895594 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5bca69-6007-41d9-bc04-7674d5605eb9 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:32.906095 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1211/2429] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:20:32.910421 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bbbe16-1f45-45e6-9be8-0a33036dfb80 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:32.920910 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bbbe16-1f45-45e6-9be8-0a33036dfb80 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:32.925452 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7147ef-fbd0-4ecc-b229-f74a4eeea533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:32.926091 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7147ef-fbd0-4ecc-b229-f74a4eeea533 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:32.926373 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7147ef-fbd0-4ecc-b229-f74a4eeea533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:32.926650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7147ef-fbd0-4ecc-b229-f74a4eeea533 admin admin] PATH_INFO: `/v3/projects/ea20f4546c4247909f62753547742814/users/9f8e3dfbf13f45fe9d279e490addb669/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:32.942065 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7147ef-fbd0-4ecc-b229-f74a4eeea533 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea20f4546c4247909f62753547742814, user_id=9f8e3dfbf13f45fe9d279e490addb669, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:32.949427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7147ef-fbd0-4ecc-b229-f74a4eeea533 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:32.962579 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1219/2430] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:20:32 2022] PUT /identity/v3/projects/ea20f4546c4247909f62753547742814/users/9f8e3dfbf13f45fe9d279e490addb669/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:32.973289 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64dab3-0a41-407e-8a4a-e557b2b6d622 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:32.973576 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64dab3-0a41-407e-8a4a-e557b2b6d622 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:32.973853 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d64dab3-0a41-407e-8a4a-e557b2b6d622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:33.001646 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-8d64dab3-0a41-407e-8a4a-e557b2b6d622 None None] MFA Rules not processed for user `9f8e3dfbf13f45fe9d279e490addb669`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:33.002842 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-8d64dab3-0a41-407e-8a4a-e557b2b6d622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:33.067625 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1212/2431] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:33.072081 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7147ef-fbd0-4ecc-b229-f74a4eeea533 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:33.136737 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7147ef-fbd0-4ecc-b229-f74a4eeea533 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:33.142065 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85786411-f79f-4638-bf9f-6e1f8add4ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:33.142665 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85786411-f79f-4638-bf9f-6e1f8add4ce3 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:33.142948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85786411-f79f-4638-bf9f-6e1f8add4ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:33.143347 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85786411-f79f-4638-bf9f-6e1f8add4ce3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:33.143987 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85786411-f79f-4638-bf9f-6e1f8add4ce3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:33.145883 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85786411-f79f-4638-bf9f-6e1f8add4ce3 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:33.151087 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:33.175707 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1220/2432] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:20:33 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:20:33.179299 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d64dab3-0a41-407e-8a4a-e557b2b6d622 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:33.189813 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d64dab3-0a41-407e-8a4a-e557b2b6d622 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:33.244936 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d051a-b6d5-4bfc-bdb8-f25c5c9556a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:33.245530 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d051a-b6d5-4bfc-bdb8-f25c5c9556a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:33.245812 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d051a-b6d5-4bfc-bdb8-f25c5c9556a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:33.246083 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d051a-b6d5-4bfc-bdb8-f25c5c9556a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:33.246718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d051a-b6d5-4bfc-bdb8-f25c5c9556a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:33.248888 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d051a-b6d5-4bfc-bdb8-f25c5c9556a9 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:33.269136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1213/2433] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:20:33 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:20:33.273235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85786411-f79f-4638-bf9f-6e1f8add4ce3 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:33.284147 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85786411-f79f-4638-bf9f-6e1f8add4ce3 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:33.288903 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9ee07-9fbd-48f8-983f-36b32447972f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:33.289501 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9ee07-9fbd-48f8-983f-36b32447972f admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:33.289785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9ee07-9fbd-48f8-983f-36b32447972f admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:33.290062 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9ee07-9fbd-48f8-983f-36b32447972f admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:33.290701 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9ee07-9fbd-48f8-983f-36b32447972f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:33.292288 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9ee07-9fbd-48f8-983f-36b32447972f admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:33.302790 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1221/2434] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:20:33.306608 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d051a-b6d5-4bfc-bdb8-f25c5c9556a9 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:33.316897 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d051a-b6d5-4bfc-bdb8-f25c5c9556a9 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:33.321575 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b6e0c-5069-4659-be09-00dbeb297630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:33.322108 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b6e0c-5069-4659-be09-00dbeb297630 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:33.322396 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b6e0c-5069-4659-be09-00dbeb297630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:33.322684 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b6e0c-5069-4659-be09-00dbeb297630 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:33.323322 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6b6e0c-5069-4659-be09-00dbeb297630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:33.324977 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6b6e0c-5069-4659-be09-00dbeb297630 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:33.334888 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1214/2435] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:20:33.338870 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9ee07-9fbd-48f8-983f-36b32447972f admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:33.349128 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9ee07-9fbd-48f8-983f-36b32447972f admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:33.353975 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38305835-c213-4740-9f0b-83b899131b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:33.354643 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38305835-c213-4740-9f0b-83b899131b02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:33.354924 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38305835-c213-4740-9f0b-83b899131b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:33.355199 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38305835-c213-4740-9f0b-83b899131b02 admin admin] PATH_INFO: `/v3/projects/293b8a27f4fc4747ae2c4c2b5f7914d6/users/168d6f902d7f4fd8811264d7635b0609/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:33.370437 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38305835-c213-4740-9f0b-83b899131b02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=293b8a27f4fc4747ae2c4c2b5f7914d6, user_id=168d6f902d7f4fd8811264d7635b0609, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:33.373191 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38305835-c213-4740-9f0b-83b899131b02 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:33.385655 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1222/2436] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:20:33 2022] PUT /identity/v3/projects/293b8a27f4fc4747ae2c4c2b5f7914d6/users/168d6f902d7f4fd8811264d7635b0609/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:33.397737 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa365df3-e3c4-4cc7-817b-3b72170633cb None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:33.398024 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa365df3-e3c4-4cc7-817b-3b72170633cb None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:33.398300 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa365df3-e3c4-4cc7-817b-3b72170633cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:33.424413 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:33.427681 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-aa365df3-e3c4-4cc7-817b-3b72170633cb None None] MFA Rules not processed for user `168d6f902d7f4fd8811264d7635b0609`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:33.429582 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-aa365df3-e3c4-4cc7-817b-3b72170633cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:33.494822 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1215/2437] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:33.501105 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38305835-c213-4740-9f0b-83b899131b02 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:33.562853 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38305835-c213-4740-9f0b-83b899131b02 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:33.567468 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581297a-5a74-4780-bf6a-a8748a7c33e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:33.568063 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581297a-5a74-4780-bf6a-a8748a7c33e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:33.568343 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581297a-5a74-4780-bf6a-a8748a7c33e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:33.568644 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581297a-5a74-4780-bf6a-a8748a7c33e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:33.569262 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581297a-5a74-4780-bf6a-a8748a7c33e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:33.571141 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581297a-5a74-4780-bf6a-a8748a7c33e0 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:33.598422 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1223/2438] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:20:33 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:20:33.602686 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa365df3-e3c4-4cc7-817b-3b72170633cb None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:33.615022 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa365df3-e3c4-4cc7-817b-3b72170633cb None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:33.674174 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5843c54-3cd1-4e65-bb9c-a0453e4f12c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:33.674899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5843c54-3cd1-4e65-bb9c-a0453e4f12c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:33.675187 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5843c54-3cd1-4e65-bb9c-a0453e4f12c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:33.675467 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5843c54-3cd1-4e65-bb9c-a0453e4f12c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:33.676110 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5843c54-3cd1-4e65-bb9c-a0453e4f12c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:33.679117 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5843c54-3cd1-4e65-bb9c-a0453e4f12c9 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:33.702341 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1216/2439] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:20:33 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:20:33.706754 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581297a-5a74-4780-bf6a-a8748a7c33e0 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:33.717544 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581297a-5a74-4780-bf6a-a8748a7c33e0 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:33.722266 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a229d4-3a94-4426-841d-68318a891e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:33.722868 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a229d4-3a94-4426-841d-68318a891e6d admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:33.723151 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a229d4-3a94-4426-841d-68318a891e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:33.723433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a229d4-3a94-4426-841d-68318a891e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:33.724072 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a229d4-3a94-4426-841d-68318a891e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:33.725693 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a229d4-3a94-4426-841d-68318a891e6d admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:33.736306 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1224/2440] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:20:33.740809 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5843c54-3cd1-4e65-bb9c-a0453e4f12c9 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:33.751220 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5843c54-3cd1-4e65-bb9c-a0453e4f12c9 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:33.755891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d24a7-a7a1-4b48-8b4a-2f5e4adecffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:33.756565 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d24a7-a7a1-4b48-8b4a-2f5e4adecffa admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:33.756850 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d24a7-a7a1-4b48-8b4a-2f5e4adecffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:33.757127 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d24a7-a7a1-4b48-8b4a-2f5e4adecffa admin admin] PATH_INFO: `/v3/projects/8770b5ce0fb84a509b60c5911c777ab4/users/ea9026145f8e40318477980ef07d4de2/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:33.772332 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d24a7-a7a1-4b48-8b4a-2f5e4adecffa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8770b5ce0fb84a509b60c5911c777ab4, user_id=ea9026145f8e40318477980ef07d4de2, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:33.774972 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7d24a7-a7a1-4b48-8b4a-2f5e4adecffa admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:33.787667 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1217/2441] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:20:33 2022] PUT /identity/v3/projects/8770b5ce0fb84a509b60c5911c777ab4/users/ea9026145f8e40318477980ef07d4de2/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:33.791733 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a229d4-3a94-4426-841d-68318a891e6d admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:33.801914 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a229d4-3a94-4426-841d-68318a891e6d admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:33.859230 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfde1ad-7ff3-42e0-85bb-ecff6a4c20f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:33.859770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfde1ad-7ff3-42e0-85bb-ecff6a4c20f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:33.860057 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfde1ad-7ff3-42e0-85bb-ecff6a4c20f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:33.860337 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfde1ad-7ff3-42e0-85bb-ecff6a4c20f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:33.861004 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfde1ad-7ff3-42e0-85bb-ecff6a4c20f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:33.862590 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfde1ad-7ff3-42e0-85bb-ecff6a4c20f8 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:33.872297 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1225/2442] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:20:33.877040 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d24a7-a7a1-4b48-8b4a-2f5e4adecffa admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:33.887414 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d24a7-a7a1-4b48-8b4a-2f5e4adecffa admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:33.892753 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3e8d0-2abc-457f-a88d-394977222aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:33.893398 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3e8d0-2abc-457f-a88d-394977222aaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:33.893682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3e8d0-2abc-457f-a88d-394977222aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:33.893961 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3e8d0-2abc-457f-a88d-394977222aaf admin admin] PATH_INFO: `/v3/projects/8770b5ce0fb84a509b60c5911c777ab4/users/ea9026145f8e40318477980ef07d4de2/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:33.895856 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3e8d0-2abc-457f-a88d-394977222aaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8770b5ce0fb84a509b60c5911c777ab4, user_id=ea9026145f8e40318477980ef07d4de2, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:33.898475 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3e8d0-2abc-457f-a88d-394977222aaf admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:33.908769 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1218/2443] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:20:33 2022] PUT /identity/v3/projects/8770b5ce0fb84a509b60c5911c777ab4/users/ea9026145f8e40318477980ef07d4de2/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:33.918659 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc2fd5-3f50-4590-979f-e9160823470b None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:33.918949 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc2fd5-3f50-4590-979f-e9160823470b None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:33.919228 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc2fd5-3f50-4590-979f-e9160823470b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:33.947058 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-d5dc2fd5-3f50-4590-979f-e9160823470b None None] MFA Rules not processed for user `ea9026145f8e40318477980ef07d4de2`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:33.948268 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d5dc2fd5-3f50-4590-979f-e9160823470b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:33.957958 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:34.017220 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1226/2444] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:33 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:34.028361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3e8d0-2abc-457f-a88d-394977222aaf admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:34.030342 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-4fe3e8d0-2abc-457f-a88d-394977222aaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:34.087393 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3e8d0-2abc-457f-a88d-394977222aaf admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:34.091102 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17bd9c-4a92-4f26-a98e-d91317ba7131 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:34.091984 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17bd9c-4a92-4f26-a98e-d91317ba7131 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:34.092295 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17bd9c-4a92-4f26-a98e-d91317ba7131 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:34.092602 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17bd9c-4a92-4f26-a98e-d91317ba7131 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:34.101862 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:34.152957 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17bd9c-4a92-4f26-a98e-d91317ba7131 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:34.154444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17bd9c-4a92-4f26-a98e-d91317ba7131 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:34.163397 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1219/2445] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:20:35.660475 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc2fd5-3f50-4590-979f-e9160823470b None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:35.661893 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d5dc2fd5-3f50-4590-979f-e9160823470b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:35.674149 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc2fd5-3f50-4590-979f-e9160823470b None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:35.677702 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e1047-4f5c-428f-b009-6cb8a5c8e440 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:35.678294 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e1047-4f5c-428f-b009-6cb8a5c8e440 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:35.678572 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e1047-4f5c-428f-b009-6cb8a5c8e440 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:35.678843 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e1047-4f5c-428f-b009-6cb8a5c8e440 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:35.735945 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e1047-4f5c-428f-b009-6cb8a5c8e440 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:35.737313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e1047-4f5c-428f-b009-6cb8a5c8e440 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:35.746192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1227/2446] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:20:36.253538 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17bd9c-4a92-4f26-a98e-d91317ba7131 service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:36.262210 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17bd9c-4a92-4f26-a98e-d91317ba7131 service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:36.265867 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace0951-a3da-4062-bdb5-ce012905ba24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:36.266457 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace0951-a3da-4062-bdb5-ce012905ba24 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:36.266736 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace0951-a3da-4062-bdb5-ce012905ba24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:36.267005 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace0951-a3da-4062-bdb5-ce012905ba24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:36.275785 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace0951-a3da-4062-bdb5-ce012905ba24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:36.277283 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace0951-a3da-4062-bdb5-ce012905ba24 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:36.287285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1220/2447] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:36 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:20:40.155547 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e1047-4f5c-428f-b009-6cb8a5c8e440 service cinder] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:40.217689 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e1047-4f5c-428f-b009-6cb8a5c8e440 service cinder] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:40.222221 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d252082-a4da-490a-982f-aff1de19b0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:40.222841 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d252082-a4da-490a-982f-aff1de19b0f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:40.223125 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d252082-a4da-490a-982f-aff1de19b0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:40.223425 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d252082-a4da-490a-982f-aff1de19b0f7 admin admin] PATH_INFO: `/v3/users/9f8e3dfbf13f45fe9d279e490addb669` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:40.224552 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d252082-a4da-490a-982f-aff1de19b0f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f8e3dfbf13f45fe9d279e490addb669)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:40.226441 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d252082-a4da-490a-982f-aff1de19b0f7 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:40.272263 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5d252082-a4da-490a-982f-aff1de19b0f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f8e3dfbf13f45fe9d279e490addb669'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:40.272864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5d252082-a4da-490a-982f-aff1de19b0f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f8e3dfbf13f45fe9d279e490addb669'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:40.274050 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:40.281915 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5d252082-a4da-490a-982f-aff1de19b0f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f8e3dfbf13f45fe9d279e490addb669'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:40.293785 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5d252082-a4da-490a-982f-aff1de19b0f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f8e3dfbf13f45fe9d279e490addb669'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:40.301364 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-5d252082-a4da-490a-982f-aff1de19b0f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f8e3dfbf13f45fe9d279e490addb669'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:40.307156 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1228/2448] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:20:40 2022] DELETE /identity/v3/users/9f8e3dfbf13f45fe9d279e490addb669 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:40.312664 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab4f28-b184-4653-9510-11a83eff8cbe None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:40.313165 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab4f28-b184-4653-9510-11a83eff8cbe None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:40.313329 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab4f28-b184-4653-9510-11a83eff8cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:40.332144 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1dab4f28-b184-4653-9510-11a83eff8cbe None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:40.333380 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1dab4f28-b184-4653-9510-11a83eff8cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:40.391309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1221/2449] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:40.400966 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d252082-a4da-490a-982f-aff1de19b0f7 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:40.402361 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-5d252082-a4da-490a-982f-aff1de19b0f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:40.459245 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d252082-a4da-490a-982f-aff1de19b0f7 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:40.462879 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973727e0-8743-4dbb-bfc9-482924a6b5ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:40.463472 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973727e0-8743-4dbb-bfc9-482924a6b5ab service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:40.463751 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973727e0-8743-4dbb-bfc9-482924a6b5ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:40.464024 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973727e0-8743-4dbb-bfc9-482924a6b5ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:40.473376 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973727e0-8743-4dbb-bfc9-482924a6b5ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:40.474782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973727e0-8743-4dbb-bfc9-482924a6b5ab service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:40.484712 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1229/2450] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:20:40.752151 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab4f28-b184-4653-9510-11a83eff8cbe None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:40.753664 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1dab4f28-b184-4653-9510-11a83eff8cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:40.765887 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab4f28-b184-4653-9510-11a83eff8cbe None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:40.771262 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d5811-9b8f-4cbf-9062-11ab80134feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:40.772096 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d5811-9b8f-4cbf-9062-11ab80134feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:40.772581 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d5811-9b8f-4cbf-9062-11ab80134feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:40.773048 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d5811-9b8f-4cbf-9062-11ab80134feb admin admin] PATH_INFO: `/v3/projects/ea20f4546c4247909f62753547742814` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:40.774379 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d5811-9b8f-4cbf-9062-11ab80134feb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea20f4546c4247909f62753547742814)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:40.776275 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d5811-9b8f-4cbf-9062-11ab80134feb admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:40.825805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-eb0d5811-9b8f-4cbf-9062-11ab80134feb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea20f4546c4247909f62753547742814', 'request_id': 'req-eb0d5811-9b8f-4cbf-9062-11ab80134feb'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:40.832098 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-eb0d5811-9b8f-4cbf-9062-11ab80134feb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea20f4546c4247909f62753547742814', 'request_id': 'req-eb0d5811-9b8f-4cbf-9062-11ab80134feb'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:40.839581 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-eb0d5811-9b8f-4cbf-9062-11ab80134feb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea20f4546c4247909f62753547742814', 'request_id': 'req-eb0d5811-9b8f-4cbf-9062-11ab80134feb'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:40.850656 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-eb0d5811-9b8f-4cbf-9062-11ab80134feb admin admin] The token cache is being invalidate because project ea20f4546c4247909f62753547742814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:20:40.850951 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-eb0d5811-9b8f-4cbf-9062-11ab80134feb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:40.851567 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-eb0d5811-9b8f-4cbf-9062-11ab80134feb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:40.853105 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1222/2451] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:20:40 2022] DELETE /identity/v3/projects/ea20f4546c4247909f62753547742814 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:40.858077 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973727e0-8743-4dbb-bfc9-482924a6b5ab service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:40.868153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973727e0-8743-4dbb-bfc9-482924a6b5ab service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:40.924968 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88397d1-f516-42b6-9b38-944e315d1f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:40.925688 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88397d1-f516-42b6-9b38-944e315d1f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:40.926009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88397d1-f516-42b6-9b38-944e315d1f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:40.926289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88397d1-f516-42b6-9b38-944e315d1f9e admin admin] PATH_INFO: `/v3/users/168d6f902d7f4fd8811264d7635b0609` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:40.927437 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88397d1-f516-42b6-9b38-944e315d1f9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=168d6f902d7f4fd8811264d7635b0609)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:40.929405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88397d1-f516-42b6-9b38-944e315d1f9e admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:40.974603 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-a88397d1-f516-42b6-9b38-944e315d1f9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '168d6f902d7f4fd8811264d7635b0609'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:40.975257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-a88397d1-f516-42b6-9b38-944e315d1f9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '168d6f902d7f4fd8811264d7635b0609'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:40.982174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-a88397d1-f516-42b6-9b38-944e315d1f9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '168d6f902d7f4fd8811264d7635b0609'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:40.997367 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:40.999973 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-a88397d1-f516-42b6-9b38-944e315d1f9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '168d6f902d7f4fd8811264d7635b0609'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:41.008637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-a88397d1-f516-42b6-9b38-944e315d1f9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '168d6f902d7f4fd8811264d7635b0609'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:41.015848 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1230/2452] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:20:40 2022] DELETE /identity/v3/users/168d6f902d7f4fd8811264d7635b0609 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:41.300228 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d5811-9b8f-4cbf-9062-11ab80134feb admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:41.301777 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-eb0d5811-9b8f-4cbf-9062-11ab80134feb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:41.315237 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d5811-9b8f-4cbf-9062-11ab80134feb admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:41.332705 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:41.374162 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b087f1-eb49-4954-add3-75a0a4c9e2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:41.374727 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b087f1-eb49-4954-add3-75a0a4c9e2cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:41.375014 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b087f1-eb49-4954-add3-75a0a4c9e2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:41.375295 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b087f1-eb49-4954-add3-75a0a4c9e2cd admin admin] PATH_INFO: `/v3/projects/293b8a27f4fc4747ae2c4c2b5f7914d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:41.376430 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b087f1-eb49-4954-add3-75a0a4c9e2cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=293b8a27f4fc4747ae2c4c2b5f7914d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:41.378352 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b087f1-eb49-4954-add3-75a0a4c9e2cd admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:41.425229 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-06b087f1-eb49-4954-add3-75a0a4c9e2cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '293b8a27f4fc4747ae2c4c2b5f7914d6', 'request_id': 'req-06b087f1-eb49-4954-add3-75a0a4c9e2cd'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:41.431079 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-06b087f1-eb49-4954-add3-75a0a4c9e2cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '293b8a27f4fc4747ae2c4c2b5f7914d6', 'request_id': 'req-06b087f1-eb49-4954-add3-75a0a4c9e2cd'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:41.438350 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-06b087f1-eb49-4954-add3-75a0a4c9e2cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '293b8a27f4fc4747ae2c4c2b5f7914d6', 'request_id': 'req-06b087f1-eb49-4954-add3-75a0a4c9e2cd'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:41.448465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-06b087f1-eb49-4954-add3-75a0a4c9e2cd admin admin] The token cache is being invalidate because project 293b8a27f4fc4747ae2c4c2b5f7914d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:20:41.448781 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-06b087f1-eb49-4954-add3-75a0a4c9e2cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:41.449403 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-06b087f1-eb49-4954-add3-75a0a4c9e2cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:41.450890 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1223/2453] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:20:41 2022] DELETE /identity/v3/projects/293b8a27f4fc4747ae2c4c2b5f7914d6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:20:41.455370 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88397d1-f516-42b6-9b38-944e315d1f9e admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:41.456837 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-a88397d1-f516-42b6-9b38-944e315d1f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:41.469882 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88397d1-f516-42b6-9b38-944e315d1f9e admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:41.526614 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf7aad-90a0-402d-be69-d291f7492125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:41.527243 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf7aad-90a0-402d-be69-d291f7492125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:41.527523 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf7aad-90a0-402d-be69-d291f7492125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:41.527796 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf7aad-90a0-402d-be69-d291f7492125 admin admin] PATH_INFO: `/v3/users/ea9026145f8e40318477980ef07d4de2` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:41.529538 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf7aad-90a0-402d-be69-d291f7492125 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea9026145f8e40318477980ef07d4de2)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:41.531514 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baf7aad-90a0-402d-be69-d291f7492125 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:41.577549 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-1baf7aad-90a0-402d-be69-d291f7492125 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea9026145f8e40318477980ef07d4de2'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:41.578213 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-1baf7aad-90a0-402d-be69-d291f7492125 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea9026145f8e40318477980ef07d4de2'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:41.585538 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-1baf7aad-90a0-402d-be69-d291f7492125 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea9026145f8e40318477980ef07d4de2'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:41.598191 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-1baf7aad-90a0-402d-be69-d291f7492125 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea9026145f8e40318477980ef07d4de2'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:41.605852 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-1baf7aad-90a0-402d-be69-d291f7492125 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea9026145f8e40318477980ef07d4de2'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:41.616066 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1231/2454] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:20:41 2022] DELETE /identity/v3/users/ea9026145f8e40318477980ef07d4de2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:42.119594 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b087f1-eb49-4954-add3-75a0a4c9e2cd admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:42.122174 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-06b087f1-eb49-4954-add3-75a0a4c9e2cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:42.143515 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b087f1-eb49-4954-add3-75a0a4c9e2cd admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:42.224247 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697b9e0-48a8-49f9-a225-bbd58ea388a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:42.224831 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697b9e0-48a8-49f9-a225-bbd58ea388a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:42.225119 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697b9e0-48a8-49f9-a225-bbd58ea388a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:42.225401 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697b9e0-48a8-49f9-a225-bbd58ea388a3 admin admin] PATH_INFO: `/v3/projects/8770b5ce0fb84a509b60c5911c777ab4` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:42.226538 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697b9e0-48a8-49f9-a225-bbd58ea388a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8770b5ce0fb84a509b60c5911c777ab4)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:42.228463 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697b9e0-48a8-49f9-a225-bbd58ea388a3 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:42.279429 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4697b9e0-48a8-49f9-a225-bbd58ea388a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8770b5ce0fb84a509b60c5911c777ab4', 'request_id': 'req-4697b9e0-48a8-49f9-a225-bbd58ea388a3'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:42.281400 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:42.287089 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4697b9e0-48a8-49f9-a225-bbd58ea388a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8770b5ce0fb84a509b60c5911c777ab4', 'request_id': 'req-4697b9e0-48a8-49f9-a225-bbd58ea388a3'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:42.299828 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4697b9e0-48a8-49f9-a225-bbd58ea388a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8770b5ce0fb84a509b60c5911c777ab4', 'request_id': 'req-4697b9e0-48a8-49f9-a225-bbd58ea388a3'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:42.313737 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4697b9e0-48a8-49f9-a225-bbd58ea388a3 admin admin] The token cache is being invalidate because project 8770b5ce0fb84a509b60c5911c777ab4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:20:42.314718 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4697b9e0-48a8-49f9-a225-bbd58ea388a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:42.315516 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-4697b9e0-48a8-49f9-a225-bbd58ea388a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:20:42.318235 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1224/2455] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:20:42 2022] DELETE /identity/v3/projects/8770b5ce0fb84a509b60c5911c777ab4 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:20:42.323064 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31107a-cc67-4818-bf82-f8a3eb42c7e2 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:42.323307 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31107a-cc67-4818-bf82-f8a3eb42c7e2 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:42.323585 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31107a-cc67-4818-bf82-f8a3eb42c7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:42.346666 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-ef31107a-cc67-4818-bf82-f8a3eb42c7e2 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:42.347989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ef31107a-cc67-4818-bf82-f8a3eb42c7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:42.424414 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1232/2456] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:42.428005 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b25078-8445-446a-a593-c168e8c6db78 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:42.428653 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b25078-8445-446a-a593-c168e8c6db78 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:42.429091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b25078-8445-446a-a593-c168e8c6db78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:42.476860 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-06b25078-8445-446a-a593-c168e8c6db78 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:42.478966 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-06b25078-8445-446a-a593-c168e8c6db78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:42.490258 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1225/2457] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:42.494257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31107a-cc67-4818-bf82-f8a3eb42c7e2 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:42.506970 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31107a-cc67-4818-bf82-f8a3eb42c7e2 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:42.514368 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d80e9a-d09e-4743-8436-3143d9b8bc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:42.515378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d80e9a-d09e-4743-8436-3143d9b8bc3f admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:42.515830 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d80e9a-d09e-4743-8436-3143d9b8bc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:42.516261 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d80e9a-d09e-4743-8436-3143d9b8bc3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:42.516958 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d80e9a-d09e-4743-8436-3143d9b8bc3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:20:42.517270 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d80e9a-d09e-4743-8436-3143d9b8bc3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:42.518938 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d80e9a-d09e-4743-8436-3143d9b8bc3f admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:42.520644 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:42.539530 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1233/2458] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:20:42.541387 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b25078-8445-446a-a593-c168e8c6db78 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:42.552621 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b25078-8445-446a-a593-c168e8c6db78 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:42.557710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d48836-cbe9-45c2-b7a0-c25a8f44f17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:42.558465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d48836-cbe9-45c2-b7a0-c25a8f44f17e admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:42.559513 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d48836-cbe9-45c2-b7a0-c25a8f44f17e admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:42.559889 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d48836-cbe9-45c2-b7a0-c25a8f44f17e admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:42.561917 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d48836-cbe9-45c2-b7a0-c25a8f44f17e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:42.564794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d48836-cbe9-45c2-b7a0-c25a8f44f17e admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:42.602793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1226/2459] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:20:42 2022] POST /identity/v3/projects => generated 365 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:20:42.603585 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d80e9a-d09e-4743-8436-3143d9b8bc3f admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:42.616758 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d80e9a-d09e-4743-8436-3143d9b8bc3f admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:42.695761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c3664-9f93-4897-aaa7-b28b2cc30fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:42.696566 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c3664-9f93-4897-aaa7-b28b2cc30fec admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:42.696948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c3664-9f93-4897-aaa7-b28b2cc30fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:42.697322 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c3664-9f93-4897-aaa7-b28b2cc30fec admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:42.698089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c3664-9f93-4897-aaa7-b28b2cc30fec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:42.700107 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c3664-9f93-4897-aaa7-b28b2cc30fec admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:42.727835 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1234/2460] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:20:42 2022] POST /identity/v3/users => generated 283 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:20:42.729384 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d48836-cbe9-45c2-b7a0-c25a8f44f17e admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:42.740657 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d48836-cbe9-45c2-b7a0-c25a8f44f17e admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:42.745634 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d226af-65db-4f97-b225-e745266e5cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:42.746340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d226af-65db-4f97-b225-e745266e5cbd admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:42.746691 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d226af-65db-4f97-b225-e745266e5cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:42.747060 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d226af-65db-4f97-b225-e745266e5cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:42.747840 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d226af-65db-4f97-b225-e745266e5cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:42.749535 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d226af-65db-4f97-b225-e745266e5cbd admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:42.760420 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1227/2461] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:20:42.764576 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c3664-9f93-4897-aaa7-b28b2cc30fec admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:42.774696 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c3664-9f93-4897-aaa7-b28b2cc30fec admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:42.780271 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc73ad59-3011-4d79-9447-db2f0ba37b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:42.780990 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73ad59-3011-4d79-9447-db2f0ba37b57 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:42.781280 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73ad59-3011-4d79-9447-db2f0ba37b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:42.781561 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73ad59-3011-4d79-9447-db2f0ba37b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:42.782243 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc73ad59-3011-4d79-9447-db2f0ba37b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:42.783876 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc73ad59-3011-4d79-9447-db2f0ba37b57 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:42.795364 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1235/2462] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:20:42.799540 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d226af-65db-4f97-b225-e745266e5cbd admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:42.809835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d226af-65db-4f97-b225-e745266e5cbd admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:42.816203 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bdd768-862a-4c85-bfe9-1b616f7e8075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:42.816905 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdd768-862a-4c85-bfe9-1b616f7e8075 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:42.817192 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdd768-862a-4c85-bfe9-1b616f7e8075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:42.817471 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdd768-862a-4c85-bfe9-1b616f7e8075 admin admin] PATH_INFO: `/v3/projects/c26083e7a93340e7bda83b0d2d225add/users/d8b544c5bde64bd09c2e54b10f7e7621/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:42.832731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bdd768-862a-4c85-bfe9-1b616f7e8075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c26083e7a93340e7bda83b0d2d225add, user_id=d8b544c5bde64bd09c2e54b10f7e7621, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:42.835361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bdd768-862a-4c85-bfe9-1b616f7e8075 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:42.849685 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1228/2463] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:20:42 2022] PUT /identity/v3/projects/c26083e7a93340e7bda83b0d2d225add/users/d8b544c5bde64bd09c2e54b10f7e7621/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:20:42.860294 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6e48c-1da8-4840-ab68-72a03393df84 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:42.860630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6e48c-1da8-4840-ab68-72a03393df84 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:42.860932 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6e48c-1da8-4840-ab68-72a03393df84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:42.889175 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-50a6e48c-1da8-4840-ab68-72a03393df84 None None] MFA Rules not processed for user `d8b544c5bde64bd09c2e54b10f7e7621`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:42.890399 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-50a6e48c-1da8-4840-ab68-72a03393df84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:42.965168 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1236/2464] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:42.977150 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bdd768-862a-4c85-bfe9-1b616f7e8075 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:42.978920 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-12bdd768-862a-4c85-bfe9-1b616f7e8075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:43.047798 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bdd768-862a-4c85-bfe9-1b616f7e8075 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:43.053518 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d31c92-9868-40e3-8fbe-0da6feb29356 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:43.054133 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d31c92-9868-40e3-8fbe-0da6feb29356 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:43.054489 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d31c92-9868-40e3-8fbe-0da6feb29356 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:43.054850 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d31c92-9868-40e3-8fbe-0da6feb29356 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:43.064953 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d31c92-9868-40e3-8fbe-0da6feb29356 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:43.066503 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d31c92-9868-40e3-8fbe-0da6feb29356 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:43.075536 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1229/2465] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:20:51.011661 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a246b57-9963-488e-9f70-11f199cce6b9 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:51.012218 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a246b57-9963-488e-9f70-11f199cce6b9 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:51.012704 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a246b57-9963-488e-9f70-11f199cce6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:51.035873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-6a246b57-9963-488e-9f70-11f199cce6b9 None None] MFA Rules not processed for user `06e2b7a78a174a5a82493f58add77988`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:20:51.037910 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-6a246b57-9963-488e-9f70-11f199cce6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:51.101193 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1237/2466] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:20:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:20:51.463002 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d31c92-9868-40e3-8fbe-0da6feb29356 service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:51.464461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-f8d31c92-9868-40e3-8fbe-0da6feb29356 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:51.522111 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d31c92-9868-40e3-8fbe-0da6feb29356 service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:51.525794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459120b-cc9e-4d59-a0e8-0b292d6f5058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:51.526399 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459120b-cc9e-4d59-a0e8-0b292d6f5058 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:51.526690 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459120b-cc9e-4d59-a0e8-0b292d6f5058 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:51.526969 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459120b-cc9e-4d59-a0e8-0b292d6f5058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:51.528743 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-3459120b-cc9e-4d59-a0e8-0b292d6f5058 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:51.538848 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3459120b-cc9e-4d59-a0e8-0b292d6f5058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:51.540248 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3459120b-cc9e-4d59-a0e8-0b292d6f5058 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:51.548222 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1230/2467] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:20:51.879824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a246b57-9963-488e-9f70-11f199cce6b9 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:20:51.881567 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-6a246b57-9963-488e-9f70-11f199cce6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:51.892378 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:20:51.959804 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a246b57-9963-488e-9f70-11f199cce6b9 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:20:51.964850 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65274ad-a83f-4a8f-9f4d-3dcf755ea126 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24108b6355414f64ba5e7bf84011eee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:20:51.965452 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65274ad-a83f-4a8f-9f4d-3dcf755ea126 service placement] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:20:51.965735 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65274ad-a83f-4a8f-9f4d-3dcf755ea126 service placement] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:20:51.966010 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65274ad-a83f-4a8f-9f4d-3dcf755ea126 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:20:51.967800 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c65274ad-a83f-4a8f-9f4d-3dcf755ea126 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:20:51.978918 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65274ad-a83f-4a8f-9f4d-3dcf755ea126 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:20:51.980472 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65274ad-a83f-4a8f-9f4d-3dcf755ea126 service placement] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:20:51.990820 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1238/2468] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:20:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:21:00.872178 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459120b-cc9e-4d59-a0e8-0b292d6f5058 service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:21:00.873904 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-3459120b-cc9e-4d59-a0e8-0b292d6f5058 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:21:00.906466 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:21:00.959167 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459120b-cc9e-4d59-a0e8-0b292d6f5058 service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:21:00.964573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535976e-85fb-48fc-a6a5-d5584f5d9dd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:21:00.965228 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535976e-85fb-48fc-a6a5-d5584f5d9dd3 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:21:00.965512 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535976e-85fb-48fc-a6a5-d5584f5d9dd3 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:21:00.965787 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535976e-85fb-48fc-a6a5-d5584f5d9dd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:21:00.967495 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-c535976e-85fb-48fc-a6a5-d5584f5d9dd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:21:00.979463 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535976e-85fb-48fc-a6a5-d5584f5d9dd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:21:00.981117 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535976e-85fb-48fc-a6a5-d5584f5d9dd3 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:21:00.992688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1231/2469] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:21:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:21:01.986089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=29 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:21:04.664661 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65274ad-a83f-4a8f-9f4d-3dcf755ea126 service placement] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:21:04.666170 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-c65274ad-a83f-4a8f-9f4d-3dcf755ea126 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:21:04.681912 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65274ad-a83f-4a8f-9f4d-3dcf755ea126 service placement] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:21:04.688554 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbcf9d1-0722-4e68-af38-fd27461203f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:21:04.689167 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbcf9d1-0722-4e68-af38-fd27461203f6 service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:21:04.689455 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbcf9d1-0722-4e68-af38-fd27461203f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:21:04.689736 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbcf9d1-0722-4e68-af38-fd27461203f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:21:04.691555 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-afbcf9d1-0722-4e68-af38-fd27461203f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:21:04.704398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbcf9d1-0722-4e68-af38-fd27461203f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:21:04.706058 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbcf9d1-0722-4e68-af38-fd27461203f6 service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:21:04.716965 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1239/2470] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:21:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:21:10.987810 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=24 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:21:14.712338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=6 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:21:25.768464 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535976e-85fb-48fc-a6a5-d5584f5d9dd3 service nova] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:21:25.829471 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535976e-85fb-48fc-a6a5-d5584f5d9dd3 service nova] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:21:25.833948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add87a3e-c01b-4b33-99b3-54aef8c66ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:21:25.834562 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add87a3e-c01b-4b33-99b3-54aef8c66ad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:21:25.834841 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add87a3e-c01b-4b33-99b3-54aef8c66ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:21:25.835111 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add87a3e-c01b-4b33-99b3-54aef8c66ad6 admin admin] PATH_INFO: `/v3/users/d8b544c5bde64bd09c2e54b10f7e7621` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:21:25.836206 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add87a3e-c01b-4b33-99b3-54aef8c66ad6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8b544c5bde64bd09c2e54b10f7e7621)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:21:25.838627 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add87a3e-c01b-4b33-99b3-54aef8c66ad6 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:21:25.884140 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-add87a3e-c01b-4b33-99b3-54aef8c66ad6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8b544c5bde64bd09c2e54b10f7e7621'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:21:25.891127 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-add87a3e-c01b-4b33-99b3-54aef8c66ad6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8b544c5bde64bd09c2e54b10f7e7621'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:21:25.903463 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-add87a3e-c01b-4b33-99b3-54aef8c66ad6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8b544c5bde64bd09c2e54b10f7e7621'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:21:25.904155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-add87a3e-c01b-4b33-99b3-54aef8c66ad6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8b544c5bde64bd09c2e54b10f7e7621'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:21:25.904793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-add87a3e-c01b-4b33-99b3-54aef8c66ad6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8b544c5bde64bd09c2e54b10f7e7621'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:21:25.920798 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1232/2471] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:21:25 2022] DELETE /identity/v3/users/d8b544c5bde64bd09c2e54b10f7e7621 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:21:25.925237 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8e17a-7f4e-45a5-bc86-def1abdcf4be None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:21:25.925734 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8e17a-7f4e-45a5-bc86-def1abdcf4be None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:21:25.925901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8e17a-7f4e-45a5-bc86-def1abdcf4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:21:25.944925 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-d4e8e17a-7f4e-45a5-bc86-def1abdcf4be None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:21:25.946188 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d4e8e17a-7f4e-45a5-bc86-def1abdcf4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:21:26.006132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1240/2472] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:21:26.016621 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add87a3e-c01b-4b33-99b3-54aef8c66ad6 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:21:26.017969 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-add87a3e-c01b-4b33-99b3-54aef8c66ad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:21:26.074649 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add87a3e-c01b-4b33-99b3-54aef8c66ad6 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:21:26.078318 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7498677-46bb-4436-b642-96a818eabb14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:21:26.078911 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7498677-46bb-4436-b642-96a818eabb14 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:21:26.079195 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7498677-46bb-4436-b642-96a818eabb14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:21:26.079477 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7498677-46bb-4436-b642-96a818eabb14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:21:26.089045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7498677-46bb-4436-b642-96a818eabb14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:21:26.090463 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7498677-46bb-4436-b642-96a818eabb14 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:21:26.099694 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1233/2473] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:21:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:21:26.370860 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8e17a-7f4e-45a5-bc86-def1abdcf4be None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:21:26.372436 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d4e8e17a-7f4e-45a5-bc86-def1abdcf4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:21:26.389616 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8e17a-7f4e-45a5-bc86-def1abdcf4be None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:21:26.394356 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f7a27-bea7-4cd9-8a6a-0ddf42129c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:21:26.394987 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f7a27-bea7-4cd9-8a6a-0ddf42129c49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:21:26.395268 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f7a27-bea7-4cd9-8a6a-0ddf42129c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:21:26.395546 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f7a27-bea7-4cd9-8a6a-0ddf42129c49 admin admin] PATH_INFO: `/v3/projects/c26083e7a93340e7bda83b0d2d225add` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:21:26.396685 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f7a27-bea7-4cd9-8a6a-0ddf42129c49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c26083e7a93340e7bda83b0d2d225add)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:21:26.398591 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f7a27-bea7-4cd9-8a6a-0ddf42129c49 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:21:26.445487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-e21f7a27-bea7-4cd9-8a6a-0ddf42129c49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c26083e7a93340e7bda83b0d2d225add', 'request_id': 'req-e21f7a27-bea7-4cd9-8a6a-0ddf42129c49'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:21:26.453273 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-e21f7a27-bea7-4cd9-8a6a-0ddf42129c49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c26083e7a93340e7bda83b0d2d225add', 'request_id': 'req-e21f7a27-bea7-4cd9-8a6a-0ddf42129c49'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:21:26.458684 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-e21f7a27-bea7-4cd9-8a6a-0ddf42129c49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c26083e7a93340e7bda83b0d2d225add', 'request_id': 'req-e21f7a27-bea7-4cd9-8a6a-0ddf42129c49'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:21:26.470115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-e21f7a27-bea7-4cd9-8a6a-0ddf42129c49 admin admin] The token cache is being invalidate because project c26083e7a93340e7bda83b0d2d225add was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:21:26.470401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-e21f7a27-bea7-4cd9-8a6a-0ddf42129c49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:21:26.471029 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-e21f7a27-bea7-4cd9-8a6a-0ddf42129c49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:21:26.472612 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1241/2474] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:21:26 2022] DELETE /identity/v3/projects/c26083e7a93340e7bda83b0d2d225add => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:21:36.095558 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:21:36.466941 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:22:12.828114 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7498677-46bb-4436-b642-96a818eabb14 service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:22:12.829550 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a7498677-46bb-4436-b642-96a818eabb14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:22:12.898440 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7498677-46bb-4436-b642-96a818eabb14 service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:22:12.956896 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31699d-039b-4f42-838b-b0d871887095 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24108b6355414f64ba5e7bf84011eee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:22:12.957496 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31699d-039b-4f42-838b-b0d871887095 service placement] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:22:12.957780 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31699d-039b-4f42-838b-b0d871887095 service placement] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:22:12.958057 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31699d-039b-4f42-838b-b0d871887095 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:22:12.959777 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-7f31699d-039b-4f42-838b-b0d871887095 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:22:12.970878 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31699d-039b-4f42-838b-b0d871887095 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:22:12.972446 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31699d-039b-4f42-838b-b0d871887095 service placement] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:22:12.982465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1234/2475] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:22:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:22:20.418355 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f7a27-bea7-4cd9-8a6a-0ddf42129c49 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:22:20.419946 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-e21f7a27-bea7-4cd9-8a6a-0ddf42129c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:22:20.492529 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f7a27-bea7-4cd9-8a6a-0ddf42129c49 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:22:20.498901 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d55c79-5d04-4113-bd1c-e3346e52b802 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:22:20.499494 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d55c79-5d04-4113-bd1c-e3346e52b802 service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:22:20.499773 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d55c79-5d04-4113-bd1c-e3346e52b802 service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:22:20.500044 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d55c79-5d04-4113-bd1c-e3346e52b802 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:22:20.501758 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f7d55c79-5d04-4113-bd1c-e3346e52b802 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:22:20.559180 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d55c79-5d04-4113-bd1c-e3346e52b802 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:22:20.560800 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d55c79-5d04-4113-bd1c-e3346e52b802 service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:22:20.569620 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1242/2476] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:22:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:22:22.977934 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=58 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:22:30.565419 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=56 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:22:36.702616 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344aa44-a460-4b4d-b8be-8317b09d8aa4 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:22:36.703135 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344aa44-a460-4b4d-b8be-8317b09d8aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:22:36.703259 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344aa44-a460-4b4d-b8be-8317b09d8aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:22:36.722899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1344aa44-a460-4b4d-b8be-8317b09d8aa4 None None] MFA Rules not processed for user `06e2b7a78a174a5a82493f58add77988`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:22:36.724171 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1344aa44-a460-4b4d-b8be-8317b09d8aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:22:36.731589 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1235/2477] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:22:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:22:46.717667 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:03.243018 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d55c79-5d04-4113-bd1c-e3346e52b802 service nova] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:03.244409 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f7d55c79-5d04-4113-bd1c-e3346e52b802 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:03.300110 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d55c79-5d04-4113-bd1c-e3346e52b802 service nova] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:03.303676 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a3e33-2111-4c8a-a6ff-ee1c9687a294 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:03.304272 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a3e33-2111-4c8a-a6ff-ee1c9687a294 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:03.304597 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a3e33-2111-4c8a-a6ff-ee1c9687a294 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:03.304880 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a3e33-2111-4c8a-a6ff-ee1c9687a294 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:03.306557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-fa8a3e33-2111-4c8a-a6ff-ee1c9687a294 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:03.319493 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a3e33-2111-4c8a-a6ff-ee1c9687a294 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:03.320946 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a3e33-2111-4c8a-a6ff-ee1c9687a294 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:03.332127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1243/2478] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:24:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:24:11.230190 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344aa44-a460-4b4d-b8be-8317b09d8aa4 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:11.231625 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1344aa44-a460-4b4d-b8be-8317b09d8aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:11.244082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344aa44-a460-4b4d-b8be-8317b09d8aa4 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:11.247802 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291c819-3a7b-490f-bff9-9cc302d9d350 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:11.248385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291c819-3a7b-490f-bff9-9cc302d9d350 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:11.248695 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291c819-3a7b-490f-bff9-9cc302d9d350 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:11.248967 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291c819-3a7b-490f-bff9-9cc302d9d350 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:11.250591 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9291c819-3a7b-490f-bff9-9cc302d9d350 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:11.264068 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9291c819-3a7b-490f-bff9-9cc302d9d350 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:11.265522 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9291c819-3a7b-490f-bff9-9cc302d9d350 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:11.277127 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1236/2479] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:24:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:24:13.326832 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=26 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:15.637790 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a3e33-2111-4c8a-a6ff-ee1c9687a294 service cinder] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:15.651226 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a3e33-2111-4c8a-a6ff-ee1c9687a294 service cinder] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:15.658199 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d0671d-1463-422f-8c0a-6c155214b01f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:15.658796 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0671d-1463-422f-8c0a-6c155214b01f service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:15.659080 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0671d-1463-422f-8c0a-6c155214b01f service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:15.659428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0671d-1463-422f-8c0a-6c155214b01f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:15.661247 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-e9d0671d-1463-422f-8c0a-6c155214b01f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:15.674550 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d0671d-1463-422f-8c0a-6c155214b01f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:15.676136 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d0671d-1463-422f-8c0a-6c155214b01f service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:15.689627 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1244/2480] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:24:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:24:20.577577 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291c819-3a7b-490f-bff9-9cc302d9d350 service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:20.578989 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9291c819-3a7b-490f-bff9-9cc302d9d350 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:20.635713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291c819-3a7b-490f-bff9-9cc302d9d350 service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:20.639422 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5633435-309c-42b0-8461-21cf11660b62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:20.640033 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5633435-309c-42b0-8461-21cf11660b62 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:20.640318 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5633435-309c-42b0-8461-21cf11660b62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:20.640630 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5633435-309c-42b0-8461-21cf11660b62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:20.642418 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a5633435-309c-42b0-8461-21cf11660b62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:20.704251 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5633435-309c-42b0-8461-21cf11660b62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:20.705697 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5633435-309c-42b0-8461-21cf11660b62 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:20.715573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1237/2481] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:24:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:24:23.824222 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d0671d-1463-422f-8c0a-6c155214b01f service nova] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:23.825908 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-e9d0671d-1463-422f-8c0a-6c155214b01f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:23.841933 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d0671d-1463-422f-8c0a-6c155214b01f service nova] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:23.849406 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d689fd-6172-4f78-bd7d-cdc0ae476997 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24108b6355414f64ba5e7bf84011eee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:23.850009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d689fd-6172-4f78-bd7d-cdc0ae476997 service placement] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:23.850297 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d689fd-6172-4f78-bd7d-cdc0ae476997 service placement] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:23.851108 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d689fd-6172-4f78-bd7d-cdc0ae476997 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:23.853166 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-d0d689fd-6172-4f78-bd7d-cdc0ae476997 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:23.865285 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d689fd-6172-4f78-bd7d-cdc0ae476997 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:23.867629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d689fd-6172-4f78-bd7d-cdc0ae476997 service placement] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:23.879729 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1245/2482] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:24:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:24:23.880873 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5633435-309c-42b0-8461-21cf11660b62 service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:23.882518 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-a5633435-309c-42b0-8461-21cf11660b62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:23.899154 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5633435-309c-42b0-8461-21cf11660b62 service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:23.904150 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b840170-fa9d-4e7c-931f-527e5c9b15dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:23.906625 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b840170-fa9d-4e7c-931f-527e5c9b15dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:23.906979 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b840170-fa9d-4e7c-931f-527e5c9b15dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:23.907353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b840170-fa9d-4e7c-931f-527e5c9b15dc admin admin] PATH_INFO: `/v3/users/06e2b7a78a174a5a82493f58add77988` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:23.908656 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b840170-fa9d-4e7c-931f-527e5c9b15dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=06e2b7a78a174a5a82493f58add77988)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:23.910792 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b840170-fa9d-4e7c-931f-527e5c9b15dc admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:23.959359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-3b840170-fa9d-4e7c-931f-527e5c9b15dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06e2b7a78a174a5a82493f58add77988'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:23.966973 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-3b840170-fa9d-4e7c-931f-527e5c9b15dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06e2b7a78a174a5a82493f58add77988'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:23.979192 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-3b840170-fa9d-4e7c-931f-527e5c9b15dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06e2b7a78a174a5a82493f58add77988'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:23.979854 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-3b840170-fa9d-4e7c-931f-527e5c9b15dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06e2b7a78a174a5a82493f58add77988'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:23.980461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-3b840170-fa9d-4e7c-931f-527e5c9b15dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06e2b7a78a174a5a82493f58add77988'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:23.993779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1238/2483] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:24:23 2022] DELETE /identity/v3/users/06e2b7a78a174a5a82493f58add77988 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:24.169521 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d689fd-6172-4f78-bd7d-cdc0ae476997 service placement] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:24.180778 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d689fd-6172-4f78-bd7d-cdc0ae476997 service placement] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:24.238790 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ef123-4e19-48e9-9d86-410afd68d5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:24.239424 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ef123-4e19-48e9-9d86-410afd68d5c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:24.239711 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ef123-4e19-48e9-9d86-410afd68d5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:24.239992 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ef123-4e19-48e9-9d86-410afd68d5c2 admin admin] PATH_INFO: `/v3/projects/1915ae97801b493fb42f8479cafd9d87` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:24.241151 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ef123-4e19-48e9-9d86-410afd68d5c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1915ae97801b493fb42f8479cafd9d87)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:24.243054 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ef123-4e19-48e9-9d86-410afd68d5c2 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:24.289428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-101ef123-4e19-48e9-9d86-410afd68d5c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1915ae97801b493fb42f8479cafd9d87', 'request_id': 'req-101ef123-4e19-48e9-9d86-410afd68d5c2'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:24.296624 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-101ef123-4e19-48e9-9d86-410afd68d5c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1915ae97801b493fb42f8479cafd9d87', 'request_id': 'req-101ef123-4e19-48e9-9d86-410afd68d5c2'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:24.302396 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-101ef123-4e19-48e9-9d86-410afd68d5c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1915ae97801b493fb42f8479cafd9d87', 'request_id': 'req-101ef123-4e19-48e9-9d86-410afd68d5c2'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:24.313382 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-101ef123-4e19-48e9-9d86-410afd68d5c2 admin admin] The token cache is being invalidate because project 1915ae97801b493fb42f8479cafd9d87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:24:24.313676 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-101ef123-4e19-48e9-9d86-410afd68d5c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:24.314312 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-101ef123-4e19-48e9-9d86-410afd68d5c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:24.315797 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1246/2484] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:24:24 2022] DELETE /identity/v3/projects/1915ae97801b493fb42f8479cafd9d87 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:24.321503 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f76564-26f5-45cc-999c-11ef9e5f6411 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:24.321794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f76564-26f5-45cc-999c-11ef9e5f6411 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:24.322071 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f76564-26f5-45cc-999c-11ef9e5f6411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:24.342251 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-56f76564-26f5-45cc-999c-11ef9e5f6411 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:24.343455 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-56f76564-26f5-45cc-999c-11ef9e5f6411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:24.347605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:24.405037 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1239/2485] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:24:24.414266 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4f200-ea9f-46e3-86e6-c47ee4e311d6 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:24.414495 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4f200-ea9f-46e3-86e6-c47ee4e311d6 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:24.414770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4f200-ea9f-46e3-86e6-c47ee4e311d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:24.434200 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-59b4f200-ea9f-46e3-86e6-c47ee4e311d6 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:24.435411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-59b4f200-ea9f-46e3-86e6-c47ee4e311d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:24.443359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1247/2486] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:24:24.447521 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f76564-26f5-45cc-999c-11ef9e5f6411 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:24.457210 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f76564-26f5-45cc-999c-11ef9e5f6411 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:24.462302 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec4871-d415-435e-91db-cc94e93d7e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:24.462903 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec4871-d415-435e-91db-cc94e93d7e16 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:24.463188 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec4871-d415-435e-91db-cc94e93d7e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:24.463562 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec4871-d415-435e-91db-cc94e93d7e16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:24.464211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec4871-d415-435e-91db-cc94e93d7e16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:24:24.464548 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec4871-d415-435e-91db-cc94e93d7e16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:24.466119 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec4871-d415-435e-91db-cc94e93d7e16 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:24.478761 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1240/2487] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:24:24.482767 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4f200-ea9f-46e3-86e6-c47ee4e311d6 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:24.492155 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4f200-ea9f-46e3-86e6-c47ee4e311d6 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:24.497109 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70abee1b-143e-4fdc-9dbc-b9d489fcda31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:24.497703 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abee1b-143e-4fdc-9dbc-b9d489fcda31 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:24.497989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abee1b-143e-4fdc-9dbc-b9d489fcda31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:24.498266 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abee1b-143e-4fdc-9dbc-b9d489fcda31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:24.498896 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70abee1b-143e-4fdc-9dbc-b9d489fcda31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:24.500818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70abee1b-143e-4fdc-9dbc-b9d489fcda31 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:24.528361 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1248/2488] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:24:24 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:24:24.532599 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec4871-d415-435e-91db-cc94e93d7e16 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:24.542664 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec4871-d415-435e-91db-cc94e93d7e16 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:24.598546 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e63b0-0fe8-4878-a908-a4e2ff5ae922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:24.599138 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e63b0-0fe8-4878-a908-a4e2ff5ae922 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:24.599419 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e63b0-0fe8-4878-a908-a4e2ff5ae922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:24.599692 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e63b0-0fe8-4878-a908-a4e2ff5ae922 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:24.600318 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e63b0-0fe8-4878-a908-a4e2ff5ae922 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:24.602918 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e63b0-0fe8-4878-a908-a4e2ff5ae922 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:24.626672 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1241/2489] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:24:24 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:24:24.627797 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70abee1b-143e-4fdc-9dbc-b9d489fcda31 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:24.639019 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70abee1b-143e-4fdc-9dbc-b9d489fcda31 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:24.643585 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ce585-37f2-4e99-8086-2aa90eda0c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:24.644174 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ce585-37f2-4e99-8086-2aa90eda0c14 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:24.644453 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ce585-37f2-4e99-8086-2aa90eda0c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:24.644757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ce585-37f2-4e99-8086-2aa90eda0c14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:24.645389 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ce585-37f2-4e99-8086-2aa90eda0c14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:24.646956 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ce585-37f2-4e99-8086-2aa90eda0c14 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:24.657996 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1249/2490] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:24:24.661861 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e63b0-0fe8-4878-a908-a4e2ff5ae922 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:24.672325 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e63b0-0fe8-4878-a908-a4e2ff5ae922 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:24.676920 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ea27c-0db2-4340-8b95-796a288eda2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:24.677506 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ea27c-0db2-4340-8b95-796a288eda2d admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:24.677784 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ea27c-0db2-4340-8b95-796a288eda2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:24.678055 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ea27c-0db2-4340-8b95-796a288eda2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:24.678713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ea27c-0db2-4340-8b95-796a288eda2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:24.680599 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ea27c-0db2-4340-8b95-796a288eda2d admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:24.690905 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1242/2491] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:24:24.695397 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ce585-37f2-4e99-8086-2aa90eda0c14 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:24.705289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ce585-37f2-4e99-8086-2aa90eda0c14 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:24.709877 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abaa648-0333-49c3-9800-3cd26c7cb7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:24.711111 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abaa648-0333-49c3-9800-3cd26c7cb7fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:24.711395 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abaa648-0333-49c3-9800-3cd26c7cb7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:24.711672 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abaa648-0333-49c3-9800-3cd26c7cb7fb admin admin] PATH_INFO: `/v3/projects/a9cdb3d5d9484d749bc0c7c3fd13049e/users/811ebd4a97cf411a9d71228dbaa06cc2/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:24.726025 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abaa648-0333-49c3-9800-3cd26c7cb7fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9cdb3d5d9484d749bc0c7c3fd13049e, user_id=811ebd4a97cf411a9d71228dbaa06cc2, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:24.728784 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abaa648-0333-49c3-9800-3cd26c7cb7fb admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:24.737743 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:24.743528 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1250/2492] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:24:24 2022] PUT /identity/v3/projects/a9cdb3d5d9484d749bc0c7c3fd13049e/users/811ebd4a97cf411a9d71228dbaa06cc2/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:24:24.754067 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141537b6-57d6-4d3f-97bc-93e8cfafa5cf None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:24.754361 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141537b6-57d6-4d3f-97bc-93e8cfafa5cf None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:24.754661 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141537b6-57d6-4d3f-97bc-93e8cfafa5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:24.783806 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-141537b6-57d6-4d3f-97bc-93e8cfafa5cf None None] MFA Rules not processed for user `811ebd4a97cf411a9d71228dbaa06cc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:24.785049 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-141537b6-57d6-4d3f-97bc-93e8cfafa5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:24.849999 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1243/2493] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:24:24.861845 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abaa648-0333-49c3-9800-3cd26c7cb7fb admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:24.863626 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-7abaa648-0333-49c3-9800-3cd26c7cb7fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:24.937104 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abaa648-0333-49c3-9800-3cd26c7cb7fb admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:24.942585 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736156b1-9140-4ac7-b154-61249076c93d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:24.943187 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736156b1-9140-4ac7-b154-61249076c93d service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:24.943473 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736156b1-9140-4ac7-b154-61249076c93d service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:24.943753 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736156b1-9140-4ac7-b154-61249076c93d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:24.952089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736156b1-9140-4ac7-b154-61249076c93d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:24.953710 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736156b1-9140-4ac7-b154-61249076c93d service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:24.962865 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1251/2494] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:24:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:24:25.174719 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141537b6-57d6-4d3f-97bc-93e8cfafa5cf None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:25.180674 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:25.238174 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141537b6-57d6-4d3f-97bc-93e8cfafa5cf None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:25.243242 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22021cb-ce79-4ffb-bb3d-8067514a41e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:25.243866 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22021cb-ce79-4ffb-bb3d-8067514a41e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:25.244152 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22021cb-ce79-4ffb-bb3d-8067514a41e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:25.244436 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22021cb-ce79-4ffb-bb3d-8067514a41e0 admin admin] PATH_INFO: `/v3/users/811ebd4a97cf411a9d71228dbaa06cc2` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:25.245597 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22021cb-ce79-4ffb-bb3d-8067514a41e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=811ebd4a97cf411a9d71228dbaa06cc2)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:25.247761 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22021cb-ce79-4ffb-bb3d-8067514a41e0 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:25.293652 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-e22021cb-ce79-4ffb-bb3d-8067514a41e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '811ebd4a97cf411a9d71228dbaa06cc2'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:25.300280 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-e22021cb-ce79-4ffb-bb3d-8067514a41e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '811ebd4a97cf411a9d71228dbaa06cc2'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:25.312656 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-e22021cb-ce79-4ffb-bb3d-8067514a41e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '811ebd4a97cf411a9d71228dbaa06cc2'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:25.313320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-e22021cb-ce79-4ffb-bb3d-8067514a41e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '811ebd4a97cf411a9d71228dbaa06cc2'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:25.313920 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-e22021cb-ce79-4ffb-bb3d-8067514a41e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '811ebd4a97cf411a9d71228dbaa06cc2'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:25.327063 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1244/2495] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:24:25 2022] DELETE /identity/v3/users/811ebd4a97cf411a9d71228dbaa06cc2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:25.331650 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2afbc6-9726-488c-9307-4f9e5380f66a None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:25.331972 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2afbc6-9726-488c-9307-4f9e5380f66a None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:25.332288 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2afbc6-9726-488c-9307-4f9e5380f66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:25.352571 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-ca2afbc6-9726-488c-9307-4f9e5380f66a None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:25.353780 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ca2afbc6-9726-488c-9307-4f9e5380f66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:25.413774 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1252/2496] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:24:25.423618 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22021cb-ce79-4ffb-bb3d-8067514a41e0 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:25.425082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e22021cb-ce79-4ffb-bb3d-8067514a41e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:25.482511 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22021cb-ce79-4ffb-bb3d-8067514a41e0 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:25.486192 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba683eb-bc8d-4ac9-9f0d-1db28e2e1807 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:25.486789 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba683eb-bc8d-4ac9-9f0d-1db28e2e1807 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:25.487074 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba683eb-bc8d-4ac9-9f0d-1db28e2e1807 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:25.487353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba683eb-bc8d-4ac9-9f0d-1db28e2e1807 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:25.497696 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba683eb-bc8d-4ac9-9f0d-1db28e2e1807 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:25.499044 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba683eb-bc8d-4ac9-9f0d-1db28e2e1807 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:25.508667 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1245/2497] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:24:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:24:25.779162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2afbc6-9726-488c-9307-4f9e5380f66a None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:25.780694 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-ca2afbc6-9726-488c-9307-4f9e5380f66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:25.793545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2afbc6-9726-488c-9307-4f9e5380f66a None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:25.798240 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaee47ef-4abf-47ba-b222-04b579f3de5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:25.798874 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee47ef-4abf-47ba-b222-04b579f3de5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:25.799158 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee47ef-4abf-47ba-b222-04b579f3de5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:25.799435 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaee47ef-4abf-47ba-b222-04b579f3de5d admin admin] PATH_INFO: `/v3/projects/a9cdb3d5d9484d749bc0c7c3fd13049e` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:25.800573 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee47ef-4abf-47ba-b222-04b579f3de5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9cdb3d5d9484d749bc0c7c3fd13049e)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:25.802477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaee47ef-4abf-47ba-b222-04b579f3de5d admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:25.851460 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-eaee47ef-4abf-47ba-b222-04b579f3de5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9cdb3d5d9484d749bc0c7c3fd13049e', 'request_id': 'req-eaee47ef-4abf-47ba-b222-04b579f3de5d'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:25.859034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-eaee47ef-4abf-47ba-b222-04b579f3de5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9cdb3d5d9484d749bc0c7c3fd13049e', 'request_id': 'req-eaee47ef-4abf-47ba-b222-04b579f3de5d'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:25.864768 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-eaee47ef-4abf-47ba-b222-04b579f3de5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9cdb3d5d9484d749bc0c7c3fd13049e', 'request_id': 'req-eaee47ef-4abf-47ba-b222-04b579f3de5d'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:25.875882 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-eaee47ef-4abf-47ba-b222-04b579f3de5d admin admin] The token cache is being invalidate because project a9cdb3d5d9484d749bc0c7c3fd13049e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:24:25.876167 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-eaee47ef-4abf-47ba-b222-04b579f3de5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:25.876801 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-eaee47ef-4abf-47ba-b222-04b579f3de5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:25.878259 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1253/2498] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:24:25 2022] DELETE /identity/v3/projects/a9cdb3d5d9484d749bc0c7c3fd13049e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:25.883299 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bba28-4986-40e8-b721-e971d3ae31d6 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:25.883595 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bba28-4986-40e8-b721-e971d3ae31d6 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:25.883914 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bba28-4986-40e8-b721-e971d3ae31d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:25.904879 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-264bba28-4986-40e8-b721-e971d3ae31d6 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:25.906087 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-264bba28-4986-40e8-b721-e971d3ae31d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:25.927194 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:25.968338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1246/2499] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:24:25.976831 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b0974-be00-4009-99cb-cd8f4ced02c8 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:25.977117 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b0974-be00-4009-99cb-cd8f4ced02c8 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:25.977394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b0974-be00-4009-99cb-cd8f4ced02c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:25.996548 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-987b0974-be00-4009-99cb-cd8f4ced02c8 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:25.997987 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-987b0974-be00-4009-99cb-cd8f4ced02c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:26.005693 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1254/2500] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:24:26.010311 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bba28-4986-40e8-b721-e971d3ae31d6 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:26.019887 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bba28-4986-40e8-b721-e971d3ae31d6 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:26.025233 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283f3a8-db72-432b-8361-86df8d862459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:26.025764 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283f3a8-db72-432b-8361-86df8d862459 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:26.026048 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283f3a8-db72-432b-8361-86df8d862459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:26.026322 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283f3a8-db72-432b-8361-86df8d862459 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:26.026966 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1283f3a8-db72-432b-8361-86df8d862459 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:24:26.027272 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1283f3a8-db72-432b-8361-86df8d862459 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:26.028884 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1283f3a8-db72-432b-8361-86df8d862459 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:26.040620 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1247/2501] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:24:26.044680 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b0974-be00-4009-99cb-cd8f4ced02c8 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:26.048915 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:26.055220 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b0974-be00-4009-99cb-cd8f4ced02c8 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:26.060763 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c47115-9e87-4bb4-8572-92713c5051c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:26.061356 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c47115-9e87-4bb4-8572-92713c5051c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:26.061635 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c47115-9e87-4bb4-8572-92713c5051c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:26.061905 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c47115-9e87-4bb4-8572-92713c5051c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:26.062526 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c47115-9e87-4bb4-8572-92713c5051c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:26.064382 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c47115-9e87-4bb4-8572-92713c5051c2 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:26.091589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1255/2502] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:24:26 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:24:26.095792 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283f3a8-db72-432b-8361-86df8d862459 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:26.106768 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283f3a8-db72-432b-8361-86df8d862459 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:26.163892 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600ce9f-b7f3-47a2-a4c6-1232ba358b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:26.164508 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600ce9f-b7f3-47a2-a4c6-1232ba358b36 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:26.164874 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600ce9f-b7f3-47a2-a4c6-1232ba358b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:26.165154 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600ce9f-b7f3-47a2-a4c6-1232ba358b36 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:26.165793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600ce9f-b7f3-47a2-a4c6-1232ba358b36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:26.167855 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600ce9f-b7f3-47a2-a4c6-1232ba358b36 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:26.188643 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1248/2503] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:24:26 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:24:26.192824 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c47115-9e87-4bb4-8572-92713c5051c2 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:26.203498 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c47115-9e87-4bb4-8572-92713c5051c2 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:26.208035 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd208cf9-6807-49f4-9933-28734630614e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:26.208661 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd208cf9-6807-49f4-9933-28734630614e admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:26.208944 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd208cf9-6807-49f4-9933-28734630614e admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:26.209217 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd208cf9-6807-49f4-9933-28734630614e admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:26.209851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd208cf9-6807-49f4-9933-28734630614e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:26.212110 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd208cf9-6807-49f4-9933-28734630614e admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:26.221768 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1256/2504] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:24:26.225539 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600ce9f-b7f3-47a2-a4c6-1232ba358b36 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:26.236217 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600ce9f-b7f3-47a2-a4c6-1232ba358b36 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:26.240843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e3e8d-a88f-41e2-9698-db3ff4a4673c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:26.241863 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e3e8d-a88f-41e2-9698-db3ff4a4673c admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:26.242149 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e3e8d-a88f-41e2-9698-db3ff4a4673c admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:26.242425 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e3e8d-a88f-41e2-9698-db3ff4a4673c admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:26.243052 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e3e8d-a88f-41e2-9698-db3ff4a4673c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:26.244645 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e3e8d-a88f-41e2-9698-db3ff4a4673c admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:26.255181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1249/2505] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:24:26.259799 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd208cf9-6807-49f4-9933-28734630614e admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:26.269671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd208cf9-6807-49f4-9933-28734630614e admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:26.274925 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0971723-607a-40ad-9d32-d54b94a4be28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:26.275619 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0971723-607a-40ad-9d32-d54b94a4be28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:26.275851 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0971723-607a-40ad-9d32-d54b94a4be28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:26.276127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0971723-607a-40ad-9d32-d54b94a4be28 admin admin] PATH_INFO: `/v3/projects/7315ca0628bd4febbbb6d0480790081b/users/29879ab0c7d54bbabc19564db87a6171/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:26.290427 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0971723-607a-40ad-9d32-d54b94a4be28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7315ca0628bd4febbbb6d0480790081b, user_id=29879ab0c7d54bbabc19564db87a6171, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:26.293100 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0971723-607a-40ad-9d32-d54b94a4be28 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:26.307184 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1257/2506] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:24:26 2022] PUT /identity/v3/projects/7315ca0628bd4febbbb6d0480790081b/users/29879ab0c7d54bbabc19564db87a6171/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:26.318168 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742466bd-88df-4d9b-a710-6cb8026befbc None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:26.318466 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742466bd-88df-4d9b-a710-6cb8026befbc None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:26.318752 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742466bd-88df-4d9b-a710-6cb8026befbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:26.348458 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-742466bd-88df-4d9b-a710-6cb8026befbc None None] MFA Rules not processed for user `29879ab0c7d54bbabc19564db87a6171`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:26.349710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-742466bd-88df-4d9b-a710-6cb8026befbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:26.413687 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1250/2507] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:24:26.418618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0971723-607a-40ad-9d32-d54b94a4be28 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:26.479898 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0971723-607a-40ad-9d32-d54b94a4be28 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:26.484468 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716e5ed-8a6e-4fbd-acb7-0d62398bc78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:26.485099 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716e5ed-8a6e-4fbd-acb7-0d62398bc78a admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:26.485384 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716e5ed-8a6e-4fbd-acb7-0d62398bc78a admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:26.485657 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716e5ed-8a6e-4fbd-acb7-0d62398bc78a admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:26.486286 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5716e5ed-8a6e-4fbd-acb7-0d62398bc78a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:26.488167 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5716e5ed-8a6e-4fbd-acb7-0d62398bc78a admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:26.514958 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1258/2508] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:24:26 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:24:26.518987 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742466bd-88df-4d9b-a710-6cb8026befbc None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:26.530640 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742466bd-88df-4d9b-a710-6cb8026befbc None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:26.548468 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:26.587455 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add06964-ba1b-487a-9885-0a3e0a30e8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:26.588057 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add06964-ba1b-487a-9885-0a3e0a30e8bc admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:26.588342 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add06964-ba1b-487a-9885-0a3e0a30e8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:26.588652 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add06964-ba1b-487a-9885-0a3e0a30e8bc admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:26.589285 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add06964-ba1b-487a-9885-0a3e0a30e8bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:26.591172 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add06964-ba1b-487a-9885-0a3e0a30e8bc admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:26.612231 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1251/2509] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:24:26 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:24:26.617005 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716e5ed-8a6e-4fbd-acb7-0d62398bc78a admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:26.628419 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716e5ed-8a6e-4fbd-acb7-0d62398bc78a admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:26.633841 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654195c7-dcbf-4c62-8fe2-498827fdbae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:26.634445 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654195c7-dcbf-4c62-8fe2-498827fdbae4 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:26.634864 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654195c7-dcbf-4c62-8fe2-498827fdbae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:26.635150 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654195c7-dcbf-4c62-8fe2-498827fdbae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:26.635792 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654195c7-dcbf-4c62-8fe2-498827fdbae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:26.637411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654195c7-dcbf-4c62-8fe2-498827fdbae4 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:26.648300 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1259/2510] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:24:26.652569 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add06964-ba1b-487a-9885-0a3e0a30e8bc admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:26.662521 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add06964-ba1b-487a-9885-0a3e0a30e8bc admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:26.667715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe6a6b-ff93-4495-9369-24c2660a247d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:26.668356 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe6a6b-ff93-4495-9369-24c2660a247d admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:26.668674 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe6a6b-ff93-4495-9369-24c2660a247d admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:26.668948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe6a6b-ff93-4495-9369-24c2660a247d admin admin] PATH_INFO: `/v3/projects/697a00cab50c4591b7834724c87f4984/users/3261adbca1d64a049b1897a81117e9a5/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:26.684390 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe6a6b-ff93-4495-9369-24c2660a247d admin admin] RBAC: Authorizing `identity:create_grant(project_id=697a00cab50c4591b7834724c87f4984, user_id=3261adbca1d64a049b1897a81117e9a5, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:26.687022 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe6a6b-ff93-4495-9369-24c2660a247d admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:26.700161 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1252/2511] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:24:26 2022] PUT /identity/v3/projects/697a00cab50c4591b7834724c87f4984/users/3261adbca1d64a049b1897a81117e9a5/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:24:26.704087 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654195c7-dcbf-4c62-8fe2-498827fdbae4 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:26.714261 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654195c7-dcbf-4c62-8fe2-498827fdbae4 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:26.769986 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e0465-8021-45ef-81da-7a66477d76ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:26.770597 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e0465-8021-45ef-81da-7a66477d76ea admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:26.770881 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e0465-8021-45ef-81da-7a66477d76ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:26.771159 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e0465-8021-45ef-81da-7a66477d76ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:26.771793 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e0465-8021-45ef-81da-7a66477d76ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:26.773584 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e0465-8021-45ef-81da-7a66477d76ea admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:26.783374 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1260/2512] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:24:26.787630 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe6a6b-ff93-4495-9369-24c2660a247d admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:26.798296 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe6a6b-ff93-4495-9369-24c2660a247d admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:26.802887 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21d390-4f11-499b-b126-383351d41837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:26.803530 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21d390-4f11-499b-b126-383351d41837 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:26.804407 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21d390-4f11-499b-b126-383351d41837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:26.804738 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21d390-4f11-499b-b126-383351d41837 admin admin] PATH_INFO: `/v3/projects/697a00cab50c4591b7834724c87f4984/users/3261adbca1d64a049b1897a81117e9a5/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:26.806658 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21d390-4f11-499b-b126-383351d41837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=697a00cab50c4591b7834724c87f4984, user_id=3261adbca1d64a049b1897a81117e9a5, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:26.809294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21d390-4f11-499b-b126-383351d41837 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:26.819265 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1253/2513] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:24:26 2022] PUT /identity/v3/projects/697a00cab50c4591b7834724c87f4984/users/3261adbca1d64a049b1897a81117e9a5/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:24:26.829248 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c14a6d-100d-4746-9d4d-432a8555c514 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:26.829541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c14a6d-100d-4746-9d4d-432a8555c514 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:26.829817 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c14a6d-100d-4746-9d4d-432a8555c514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:26.858332 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-13c14a6d-100d-4746-9d4d-432a8555c514 None None] MFA Rules not processed for user `3261adbca1d64a049b1897a81117e9a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:26.859543 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-13c14a6d-100d-4746-9d4d-432a8555c514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:26.863458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:26.929625 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1261/2514] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:24:26.940915 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21d390-4f11-499b-b126-383351d41837 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:26.942601 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-8c21d390-4f11-499b-b126-383351d41837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:27.000297 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21d390-4f11-499b-b126-383351d41837 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:27.004216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80add2ab-5ad5-4c05-a3d5-5b1baafff336 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:27.004852 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80add2ab-5ad5-4c05-a3d5-5b1baafff336 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:27.005133 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80add2ab-5ad5-4c05-a3d5-5b1baafff336 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:27.005406 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80add2ab-5ad5-4c05-a3d5-5b1baafff336 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:27.015053 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80add2ab-5ad5-4c05-a3d5-5b1baafff336 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:27.016465 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80add2ab-5ad5-4c05-a3d5-5b1baafff336 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:27.026609 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1254/2515] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:24:26 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:24:27.409941 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c14a6d-100d-4746-9d4d-432a8555c514 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:27.471543 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c14a6d-100d-4746-9d4d-432a8555c514 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:27.476102 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254b57b-1322-4cf5-9f9c-47899e6e439e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:27.476751 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254b57b-1322-4cf5-9f9c-47899e6e439e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:27.477036 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254b57b-1322-4cf5-9f9c-47899e6e439e admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:27.477313 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254b57b-1322-4cf5-9f9c-47899e6e439e admin admin] PATH_INFO: `/v3/users/29879ab0c7d54bbabc19564db87a6171` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:27.478420 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7254b57b-1322-4cf5-9f9c-47899e6e439e admin admin] RBAC: Authorizing `identity:delete_user(user_id=29879ab0c7d54bbabc19564db87a6171)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:27.480323 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7254b57b-1322-4cf5-9f9c-47899e6e439e admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:27.528989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-7254b57b-1322-4cf5-9f9c-47899e6e439e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29879ab0c7d54bbabc19564db87a6171'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:27.529643 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-7254b57b-1322-4cf5-9f9c-47899e6e439e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29879ab0c7d54bbabc19564db87a6171'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:27.536544 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-7254b57b-1322-4cf5-9f9c-47899e6e439e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29879ab0c7d54bbabc19564db87a6171'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:27.550976 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-7254b57b-1322-4cf5-9f9c-47899e6e439e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29879ab0c7d54bbabc19564db87a6171'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:27.558929 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-7254b57b-1322-4cf5-9f9c-47899e6e439e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29879ab0c7d54bbabc19564db87a6171'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:27.565250 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1262/2516] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:24:27 2022] DELETE /identity/v3/users/29879ab0c7d54bbabc19564db87a6171 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:27.569875 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523c972-3020-42bd-8aac-9a66843a4e5d None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:27.570375 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523c972-3020-42bd-8aac-9a66843a4e5d None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:27.570483 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523c972-3020-42bd-8aac-9a66843a4e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:27.588898 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-5523c972-3020-42bd-8aac-9a66843a4e5d None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:27.590091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5523c972-3020-42bd-8aac-9a66843a4e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:27.648786 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1255/2517] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:24:27.658505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254b57b-1322-4cf5-9f9c-47899e6e439e admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:27.659951 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-7254b57b-1322-4cf5-9f9c-47899e6e439e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:27.683091 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:27.718675 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254b57b-1322-4cf5-9f9c-47899e6e439e admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:27.722359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e0f25-ddff-4def-a801-a7683326c2a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:27.722957 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e0f25-ddff-4def-a801-a7683326c2a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:27.723240 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e0f25-ddff-4def-a801-a7683326c2a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:27.723518 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e0f25-ddff-4def-a801-a7683326c2a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:27.733404 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e0f25-ddff-4def-a801-a7683326c2a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:27.734808 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e0f25-ddff-4def-a801-a7683326c2a6 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:27.744689 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1263/2518] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:24:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:24:28.008049 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523c972-3020-42bd-8aac-9a66843a4e5d None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:28.009492 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5523c972-3020-42bd-8aac-9a66843a4e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:28.021492 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523c972-3020-42bd-8aac-9a66843a4e5d None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:28.026996 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a160d-bd98-417f-8485-0569dd841dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:28.027637 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a160d-bd98-417f-8485-0569dd841dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:28.027923 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a160d-bd98-417f-8485-0569dd841dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:28.028201 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a160d-bd98-417f-8485-0569dd841dc7 admin admin] PATH_INFO: `/v3/projects/7315ca0628bd4febbbb6d0480790081b` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:28.029374 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a160d-bd98-417f-8485-0569dd841dc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7315ca0628bd4febbbb6d0480790081b)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:28.031275 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a160d-bd98-417f-8485-0569dd841dc7 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:28.043986 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:28.079698 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-df7a160d-bd98-417f-8485-0569dd841dc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7315ca0628bd4febbbb6d0480790081b', 'request_id': 'req-df7a160d-bd98-417f-8485-0569dd841dc7'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:28.086394 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-df7a160d-bd98-417f-8485-0569dd841dc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7315ca0628bd4febbbb6d0480790081b', 'request_id': 'req-df7a160d-bd98-417f-8485-0569dd841dc7'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:28.093455 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-df7a160d-bd98-417f-8485-0569dd841dc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7315ca0628bd4febbbb6d0480790081b', 'request_id': 'req-df7a160d-bd98-417f-8485-0569dd841dc7'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:28.105258 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-df7a160d-bd98-417f-8485-0569dd841dc7 admin admin] The token cache is being invalidate because project 7315ca0628bd4febbbb6d0480790081b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:24:28.105573 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-df7a160d-bd98-417f-8485-0569dd841dc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:28.106254 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-df7a160d-bd98-417f-8485-0569dd841dc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:28.108018 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1256/2519] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:24:28 2022] DELETE /identity/v3/projects/7315ca0628bd4febbbb6d0480790081b => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:28.111989 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e0f25-ddff-4def-a801-a7683326c2a6 service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:28.122043 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e0f25-ddff-4def-a801-a7683326c2a6 service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:28.178415 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a3ed4-b290-4914-845e-deb38b1476da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:28.179405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a3ed4-b290-4914-845e-deb38b1476da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:28.179653 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a3ed4-b290-4914-845e-deb38b1476da admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:28.179826 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a3ed4-b290-4914-845e-deb38b1476da admin admin] PATH_INFO: `/v3/users/3261adbca1d64a049b1897a81117e9a5` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:28.180981 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08a3ed4-b290-4914-845e-deb38b1476da admin admin] RBAC: Authorizing `identity:delete_user(user_id=3261adbca1d64a049b1897a81117e9a5)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:28.182884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08a3ed4-b290-4914-845e-deb38b1476da admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:28.231037 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b08a3ed4-b290-4914-845e-deb38b1476da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3261adbca1d64a049b1897a81117e9a5'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:28.231845 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b08a3ed4-b290-4914-845e-deb38b1476da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3261adbca1d64a049b1897a81117e9a5'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:28.240199 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b08a3ed4-b290-4914-845e-deb38b1476da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3261adbca1d64a049b1897a81117e9a5'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:28.252433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b08a3ed4-b290-4914-845e-deb38b1476da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3261adbca1d64a049b1897a81117e9a5'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:28.260398 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b08a3ed4-b290-4914-845e-deb38b1476da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3261adbca1d64a049b1897a81117e9a5'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:28.266360 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1264/2520] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:24:28 2022] DELETE /identity/v3/users/3261adbca1d64a049b1897a81117e9a5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:28.541291 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a160d-bd98-417f-8485-0569dd841dc7 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:28.543360 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-df7a160d-bd98-417f-8485-0569dd841dc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:28.558611 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a160d-bd98-417f-8485-0569dd841dc7 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:28.615262 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a6424-0d6c-4b68-b634-c6b015ba4e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:28.615916 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a6424-0d6c-4b68-b634-c6b015ba4e5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:28.616082 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a6424-0d6c-4b68-b634-c6b015ba4e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:28.616752 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a6424-0d6c-4b68-b634-c6b015ba4e5d admin admin] PATH_INFO: `/v3/projects/697a00cab50c4591b7834724c87f4984` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:28.617870 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a6424-0d6c-4b68-b634-c6b015ba4e5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=697a00cab50c4591b7834724c87f4984)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:28.619748 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a6424-0d6c-4b68-b634-c6b015ba4e5d admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:28.665441 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-790a6424-0d6c-4b68-b634-c6b015ba4e5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '697a00cab50c4591b7834724c87f4984', 'request_id': 'req-790a6424-0d6c-4b68-b634-c6b015ba4e5d'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:28.671424 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-790a6424-0d6c-4b68-b634-c6b015ba4e5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '697a00cab50c4591b7834724c87f4984', 'request_id': 'req-790a6424-0d6c-4b68-b634-c6b015ba4e5d'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:28.678565 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-790a6424-0d6c-4b68-b634-c6b015ba4e5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '697a00cab50c4591b7834724c87f4984', 'request_id': 'req-790a6424-0d6c-4b68-b634-c6b015ba4e5d'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:28.689938 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-790a6424-0d6c-4b68-b634-c6b015ba4e5d admin admin] The token cache is being invalidate because project 697a00cab50c4591b7834724c87f4984 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:24:28.690225 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-790a6424-0d6c-4b68-b634-c6b015ba4e5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:28.690858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-790a6424-0d6c-4b68-b634-c6b015ba4e5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:28.692362 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1257/2521] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:24:28 2022] DELETE /identity/v3/projects/697a00cab50c4591b7834724c87f4984 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:28.697968 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a2ff2-cb45-4daf-907e-93661f9a3040 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:28.698263 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a2ff2-cb45-4daf-907e-93661f9a3040 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:28.698544 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a2ff2-cb45-4daf-907e-93661f9a3040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:28.718274 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-145a2ff2-cb45-4daf-907e-93661f9a3040 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:28.719477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-145a2ff2-cb45-4daf-907e-93661f9a3040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:28.758433 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:28.780825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1265/2522] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:24:28.788951 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5e345-6614-41f3-a390-f343c82e9970 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:28.789248 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5e345-6614-41f3-a390-f343c82e9970 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:28.789533 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5e345-6614-41f3-a390-f343c82e9970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:28.809520 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-6bf5e345-6614-41f3-a390-f343c82e9970 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:28.810723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-6bf5e345-6614-41f3-a390-f343c82e9970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:28.818686 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1258/2523] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:24:28.823384 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a2ff2-cb45-4daf-907e-93661f9a3040 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:28.833972 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a2ff2-cb45-4daf-907e-93661f9a3040 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:28.838802 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179f045-57a7-44a2-8e1d-92f359d2a6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:28.839408 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179f045-57a7-44a2-8e1d-92f359d2a6ad admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:28.839690 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179f045-57a7-44a2-8e1d-92f359d2a6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:28.839966 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179f045-57a7-44a2-8e1d-92f359d2a6ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:28.840662 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179f045-57a7-44a2-8e1d-92f359d2a6ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81380) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:24:28.840974 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179f045-57a7-44a2-8e1d-92f359d2a6ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:28.842563 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179f045-57a7-44a2-8e1d-92f359d2a6ad admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:28.854265 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1266/2524] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:24:28.858480 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5e345-6614-41f3-a390-f343c82e9970 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:28.868145 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5e345-6614-41f3-a390-f343c82e9970 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:28.872999 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c1e69-2438-4d20-8d69-612bb5e5afef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:28.873584 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c1e69-2438-4d20-8d69-612bb5e5afef admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:28.873861 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c1e69-2438-4d20-8d69-612bb5e5afef admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:28.874133 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c1e69-2438-4d20-8d69-612bb5e5afef admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:28.874753 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6c1e69-2438-4d20-8d69-612bb5e5afef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:28.876651 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6c1e69-2438-4d20-8d69-612bb5e5afef admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:28.903910 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1259/2525] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:24:28 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:24:28.907948 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179f045-57a7-44a2-8e1d-92f359d2a6ad admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:28.918005 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179f045-57a7-44a2-8e1d-92f359d2a6ad admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:28.973841 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01f714-0b69-46c2-a109-5c49e2254f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:28.974439 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01f714-0b69-46c2-a109-5c49e2254f45 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:28.974717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01f714-0b69-46c2-a109-5c49e2254f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:28.974992 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01f714-0b69-46c2-a109-5c49e2254f45 admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:28.975620 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af01f714-0b69-46c2-a109-5c49e2254f45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:28.977603 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af01f714-0b69-46c2-a109-5c49e2254f45 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:28.998582 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1267/2526] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:24:28 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:24:29.002762 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c1e69-2438-4d20-8d69-612bb5e5afef admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:29.008786 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:29.015084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c1e69-2438-4d20-8d69-612bb5e5afef admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:29.020837 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70089bd0-6db5-4581-b96a-e05bdc59589a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:29.021461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70089bd0-6db5-4581-b96a-e05bdc59589a admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:29.021742 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70089bd0-6db5-4581-b96a-e05bdc59589a admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:29.022017 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70089bd0-6db5-4581-b96a-e05bdc59589a admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:29.022677 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70089bd0-6db5-4581-b96a-e05bdc59589a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:29.024269 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70089bd0-6db5-4581-b96a-e05bdc59589a admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:29.034309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1260/2527] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:24:29.038712 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01f714-0b69-46c2-a109-5c49e2254f45 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:29.048793 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01f714-0b69-46c2-a109-5c49e2254f45 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:29.053446 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe7a3e-f94e-4584-ae01-4c3c0a3d9d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:29.054720 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe7a3e-f94e-4584-ae01-4c3c0a3d9d28 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:29.055010 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe7a3e-f94e-4584-ae01-4c3c0a3d9d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:29.055291 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe7a3e-f94e-4584-ae01-4c3c0a3d9d28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:29.055931 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe7a3e-f94e-4584-ae01-4c3c0a3d9d28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:29.057545 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe7a3e-f94e-4584-ae01-4c3c0a3d9d28 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:29.067919 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1268/2528] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:24:29.072192 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70089bd0-6db5-4581-b96a-e05bdc59589a admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:29.081905 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70089bd0-6db5-4581-b96a-e05bdc59589a admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:29.086565 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cf2f0-fad6-4b46-b88b-f49aa6dc6446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:29.087205 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cf2f0-fad6-4b46-b88b-f49aa6dc6446 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:29.087488 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cf2f0-fad6-4b46-b88b-f49aa6dc6446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:29.087763 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9cf2f0-fad6-4b46-b88b-f49aa6dc6446 admin admin] PATH_INFO: `/v3/projects/5cc46311c2d442e88167356a3f0023a6/users/a32e506aca734128a8064bba7484cc34/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:29.102359 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9cf2f0-fad6-4b46-b88b-f49aa6dc6446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cc46311c2d442e88167356a3f0023a6, user_id=a32e506aca734128a8064bba7484cc34, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:29.104996 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9cf2f0-fad6-4b46-b88b-f49aa6dc6446 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:29.118941 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1261/2529] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:24:29 2022] PUT /identity/v3/projects/5cc46311c2d442e88167356a3f0023a6/users/a32e506aca734128a8064bba7484cc34/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:24:29.182117 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa78011-2ae8-4e27-9c29-31f7859de09e None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:29.182704 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa78011-2ae8-4e27-9c29-31f7859de09e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:29.182704 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa78011-2ae8-4e27-9c29-31f7859de09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:29.210309 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-daa78011-2ae8-4e27-9c29-31f7859de09e None None] MFA Rules not processed for user `a32e506aca734128a8064bba7484cc34`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:29.211507 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-daa78011-2ae8-4e27-9c29-31f7859de09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:29.275474 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1269/2530] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:24:29.279924 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cf2f0-fad6-4b46-b88b-f49aa6dc6446 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:29.341638 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9cf2f0-fad6-4b46-b88b-f49aa6dc6446 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:29.346273 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1617b5-1bd7-4f48-8d7e-92646be2badf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:29.346869 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1617b5-1bd7-4f48-8d7e-92646be2badf admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:29.347154 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1617b5-1bd7-4f48-8d7e-92646be2badf admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:29.347431 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1617b5-1bd7-4f48-8d7e-92646be2badf admin admin] PATH_INFO: `/v3/projects` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:29.348059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1617b5-1bd7-4f48-8d7e-92646be2badf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:29.349956 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1617b5-1bd7-4f48-8d7e-92646be2badf admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:29.376468 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1262/2531] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:24:29 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:24:29.380147 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa78011-2ae8-4e27-9c29-31f7859de09e None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:29.391507 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa78011-2ae8-4e27-9c29-31f7859de09e None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:29.424703 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:29.449389 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10919b93-4461-4386-b32c-95a834c24c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:29.449998 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10919b93-4461-4386-b32c-95a834c24c0e admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:29.450292 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10919b93-4461-4386-b32c-95a834c24c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:29.450605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10919b93-4461-4386-b32c-95a834c24c0e admin admin] PATH_INFO: `/v3/users` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:29.451246 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10919b93-4461-4386-b32c-95a834c24c0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:29.453239 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10919b93-4461-4386-b32c-95a834c24c0e admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:29.473719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1270/2532] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:24:29 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:24:29.477765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1617b5-1bd7-4f48-8d7e-92646be2badf admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:29.488143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1617b5-1bd7-4f48-8d7e-92646be2badf admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:29.493268 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307fe904-61bc-4ea7-b01c-3406ab248a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:29.493851 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fe904-61bc-4ea7-b01c-3406ab248a0d admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:29.494132 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fe904-61bc-4ea7-b01c-3406ab248a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:29.494404 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fe904-61bc-4ea7-b01c-3406ab248a0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:29.495030 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307fe904-61bc-4ea7-b01c-3406ab248a0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:29.496635 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307fe904-61bc-4ea7-b01c-3406ab248a0d admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:29.507734 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1263/2533] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:24:29.511761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10919b93-4461-4386-b32c-95a834c24c0e admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:29.521401 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10919b93-4461-4386-b32c-95a834c24c0e admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:29.526106 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fcf6b-63d5-430c-a062-935401df2dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:29.526742 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fcf6b-63d5-430c-a062-935401df2dfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:29.527025 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fcf6b-63d5-430c-a062-935401df2dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:29.527300 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693fcf6b-63d5-430c-a062-935401df2dfc admin admin] PATH_INFO: `/v3/projects/f15ce62009c64053a42a3d1f9d7e7630/users/b5550ccf72b14edc844c71e97fcebfec/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:29.542011 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693fcf6b-63d5-430c-a062-935401df2dfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f15ce62009c64053a42a3d1f9d7e7630, user_id=b5550ccf72b14edc844c71e97fcebfec, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:29.544859 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693fcf6b-63d5-430c-a062-935401df2dfc admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:29.557798 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1271/2534] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:24:29 2022] PUT /identity/v3/projects/f15ce62009c64053a42a3d1f9d7e7630/users/b5550ccf72b14edc844c71e97fcebfec/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:29.561730 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307fe904-61bc-4ea7-b01c-3406ab248a0d admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:29.572287 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307fe904-61bc-4ea7-b01c-3406ab248a0d admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:29.629759 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f5eac-d5e7-4162-a244-4f024fde90c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:29.630367 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f5eac-d5e7-4162-a244-4f024fde90c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:29.630568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f5eac-d5e7-4162-a244-4f024fde90c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:29.630845 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f5eac-d5e7-4162-a244-4f024fde90c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:29.631480 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f5eac-d5e7-4162-a244-4f024fde90c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:29.633896 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f5eac-d5e7-4162-a244-4f024fde90c9 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:29.643717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1264/2535] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:24:29.648825 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fcf6b-63d5-430c-a062-935401df2dfc admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:29.659089 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693fcf6b-63d5-430c-a062-935401df2dfc admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:29.664396 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5feb62c-1f11-46f7-98ed-795222760b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:29.665068 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5feb62c-1f11-46f7-98ed-795222760b75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:29.665351 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5feb62c-1f11-46f7-98ed-795222760b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:29.665626 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5feb62c-1f11-46f7-98ed-795222760b75 admin admin] PATH_INFO: `/v3/projects/f15ce62009c64053a42a3d1f9d7e7630/users/b5550ccf72b14edc844c71e97fcebfec/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:29.667528 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5feb62c-1f11-46f7-98ed-795222760b75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f15ce62009c64053a42a3d1f9d7e7630, user_id=b5550ccf72b14edc844c71e97fcebfec, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:29.670184 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5feb62c-1f11-46f7-98ed-795222760b75 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:29.680289 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1272/2536] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:24:29 2022] PUT /identity/v3/projects/f15ce62009c64053a42a3d1f9d7e7630/users/b5550ccf72b14edc844c71e97fcebfec/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:29.690066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d90e2c-6661-4bb1-83c1-7aadc893ded9 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:29.690355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d90e2c-6661-4bb1-83c1-7aadc893ded9 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:29.690658 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d90e2c-6661-4bb1-83c1-7aadc893ded9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:29.718863 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-e9d90e2c-6661-4bb1-83c1-7aadc893ded9 None None] MFA Rules not processed for user `b5550ccf72b14edc844c71e97fcebfec`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:29.720061 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-e9d90e2c-6661-4bb1-83c1-7aadc893ded9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:29.746958 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:29.789817 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1265/2537] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:29 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:24:29.801073 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5feb62c-1f11-46f7-98ed-795222760b75 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:29.802740 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-e5feb62c-1f11-46f7-98ed-795222760b75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:29.861553 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5feb62c-1f11-46f7-98ed-795222760b75 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:29.865283 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa878e82-7ffd-4d02-97b3-b453fbed1eb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:29.865869 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa878e82-7ffd-4d02-97b3-b453fbed1eb6 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:29.866152 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa878e82-7ffd-4d02-97b3-b453fbed1eb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:29.866428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa878e82-7ffd-4d02-97b3-b453fbed1eb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:29.875605 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa878e82-7ffd-4d02-97b3-b453fbed1eb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:29.877058 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa878e82-7ffd-4d02-97b3-b453fbed1eb6 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:29.886964 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1273/2538] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:24:29 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:24:29.953158 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d90e2c-6661-4bb1-83c1-7aadc893ded9 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:30.014335 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d90e2c-6661-4bb1-83c1-7aadc893ded9 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:30.018837 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0e11d-b355-4709-af21-92ef22cc1756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:30.019456 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0e11d-b355-4709-af21-92ef22cc1756 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:30.019737 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0e11d-b355-4709-af21-92ef22cc1756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:30.020012 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0e11d-b355-4709-af21-92ef22cc1756 admin admin] PATH_INFO: `/v3/users/a32e506aca734128a8064bba7484cc34` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:30.021138 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d0e11d-b355-4709-af21-92ef22cc1756 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a32e506aca734128a8064bba7484cc34)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:30.023835 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d0e11d-b355-4709-af21-92ef22cc1756 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:30.070845 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-03d0e11d-b355-4709-af21-92ef22cc1756 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a32e506aca734128a8064bba7484cc34'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:30.077712 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-03d0e11d-b355-4709-af21-92ef22cc1756 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a32e506aca734128a8064bba7484cc34'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:30.090353 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-03d0e11d-b355-4709-af21-92ef22cc1756 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a32e506aca734128a8064bba7484cc34'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:30.091012 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-03d0e11d-b355-4709-af21-92ef22cc1756 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a32e506aca734128a8064bba7484cc34'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:30.091618 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-03d0e11d-b355-4709-af21-92ef22cc1756 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a32e506aca734128a8064bba7484cc34'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:30.112294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1266/2539] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:24:29 2022] DELETE /identity/v3/users/a32e506aca734128a8064bba7484cc34 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:30.116849 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0185cc-a3ea-47db-b15f-676d9e55db9e None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:30.117139 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0185cc-a3ea-47db-b15f-676d9e55db9e None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:30.117418 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0185cc-a3ea-47db-b15f-676d9e55db9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:30.136886 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-8e0185cc-a3ea-47db-b15f-676d9e55db9e None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:30.138120 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8e0185cc-a3ea-47db-b15f-676d9e55db9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:30.196619 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1274/2540] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:24:30.206338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0e11d-b355-4709-af21-92ef22cc1756 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:30.207734 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-03d0e11d-b355-4709-af21-92ef22cc1756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:30.253227 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:30.266936 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0e11d-b355-4709-af21-92ef22cc1756 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:30.270565 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2bffd-334c-4b00-94ad-43fe4e85082e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:30.271170 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2bffd-334c-4b00-94ad-43fe4e85082e service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:30.271452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2bffd-334c-4b00-94ad-43fe4e85082e service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:30.271730 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d2bffd-334c-4b00-94ad-43fe4e85082e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:30.280978 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d2bffd-334c-4b00-94ad-43fe4e85082e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:30.282396 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d2bffd-334c-4b00-94ad-43fe4e85082e service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:30.292036 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1267/2541] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:24:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:24:30.559030 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0185cc-a3ea-47db-b15f-676d9e55db9e None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:30.560507 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8e0185cc-a3ea-47db-b15f-676d9e55db9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:30.574252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0185cc-a3ea-47db-b15f-676d9e55db9e None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:30.578921 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91164edc-383a-41be-b2a3-dcca1ec1d8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:30.579598 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91164edc-383a-41be-b2a3-dcca1ec1d8be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:30.579886 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91164edc-383a-41be-b2a3-dcca1ec1d8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:30.580169 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91164edc-383a-41be-b2a3-dcca1ec1d8be admin admin] PATH_INFO: `/v3/projects/5cc46311c2d442e88167356a3f0023a6` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:30.581361 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91164edc-383a-41be-b2a3-dcca1ec1d8be admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cc46311c2d442e88167356a3f0023a6)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:30.583252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91164edc-383a-41be-b2a3-dcca1ec1d8be admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:30.615406 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:30.633366 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-91164edc-383a-41be-b2a3-dcca1ec1d8be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cc46311c2d442e88167356a3f0023a6', 'request_id': 'req-91164edc-383a-41be-b2a3-dcca1ec1d8be'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:30.640716 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-91164edc-383a-41be-b2a3-dcca1ec1d8be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cc46311c2d442e88167356a3f0023a6', 'request_id': 'req-91164edc-383a-41be-b2a3-dcca1ec1d8be'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:30.646265 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-91164edc-383a-41be-b2a3-dcca1ec1d8be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cc46311c2d442e88167356a3f0023a6', 'request_id': 'req-91164edc-383a-41be-b2a3-dcca1ec1d8be'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:30.657322 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-91164edc-383a-41be-b2a3-dcca1ec1d8be admin admin] The token cache is being invalidate because project 5cc46311c2d442e88167356a3f0023a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:24:30.657611 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-91164edc-383a-41be-b2a3-dcca1ec1d8be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:30.658236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-91164edc-383a-41be-b2a3-dcca1ec1d8be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:30.659697 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1275/2542] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:24:30 2022] DELETE /identity/v3/projects/5cc46311c2d442e88167356a3f0023a6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:30.664395 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2bffd-334c-4b00-94ad-43fe4e85082e service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:30.674049 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d2bffd-334c-4b00-94ad-43fe4e85082e service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:30.729688 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac77c53-ca4d-452b-b040-87578fdc3613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:30.730307 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac77c53-ca4d-452b-b040-87578fdc3613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:30.730587 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac77c53-ca4d-452b-b040-87578fdc3613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:30.730863 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac77c53-ca4d-452b-b040-87578fdc3613 admin admin] PATH_INFO: `/v3/users/b5550ccf72b14edc844c71e97fcebfec` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:30.731973 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac77c53-ca4d-452b-b040-87578fdc3613 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5550ccf72b14edc844c71e97fcebfec)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:30.734253 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac77c53-ca4d-452b-b040-87578fdc3613 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:30.779782 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-eac77c53-ca4d-452b-b040-87578fdc3613 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5550ccf72b14edc844c71e97fcebfec'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:30.786550 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-eac77c53-ca4d-452b-b040-87578fdc3613 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5550ccf72b14edc844c71e97fcebfec'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:30.798841 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-eac77c53-ca4d-452b-b040-87578fdc3613 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5550ccf72b14edc844c71e97fcebfec'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:30.799503 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-eac77c53-ca4d-452b-b040-87578fdc3613 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5550ccf72b14edc844c71e97fcebfec'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:30.800108 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-eac77c53-ca4d-452b-b040-87578fdc3613 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5550ccf72b14edc844c71e97fcebfec'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:30.813302 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1268/2543] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:24:30 2022] DELETE /identity/v3/users/b5550ccf72b14edc844c71e97fcebfec => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:31.086074 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91164edc-383a-41be-b2a3-dcca1ec1d8be admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:31.087224 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-91164edc-383a-41be-b2a3-dcca1ec1d8be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:31.101314 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91164edc-383a-41be-b2a3-dcca1ec1d8be admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:31.165665 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61afedf-e9eb-4e14-8dd3-71a55f6417b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:31.166221 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61afedf-e9eb-4e14-8dd3-71a55f6417b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:31.166505 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61afedf-e9eb-4e14-8dd3-71a55f6417b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:31.166783 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61afedf-e9eb-4e14-8dd3-71a55f6417b3 admin admin] PATH_INFO: `/v3/projects/f15ce62009c64053a42a3d1f9d7e7630` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:31.167889 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61afedf-e9eb-4e14-8dd3-71a55f6417b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f15ce62009c64053a42a3d1f9d7e7630)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:31.169813 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61afedf-e9eb-4e14-8dd3-71a55f6417b3 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:31.215110 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-d61afedf-e9eb-4e14-8dd3-71a55f6417b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f15ce62009c64053a42a3d1f9d7e7630', 'request_id': 'req-d61afedf-e9eb-4e14-8dd3-71a55f6417b3'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:31.222005 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-d61afedf-e9eb-4e14-8dd3-71a55f6417b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f15ce62009c64053a42a3d1f9d7e7630', 'request_id': 'req-d61afedf-e9eb-4e14-8dd3-71a55f6417b3'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:31.228905 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-d61afedf-e9eb-4e14-8dd3-71a55f6417b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f15ce62009c64053a42a3d1f9d7e7630', 'request_id': 'req-d61afedf-e9eb-4e14-8dd3-71a55f6417b3'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:31.239618 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-d61afedf-e9eb-4e14-8dd3-71a55f6417b3 admin admin] The token cache is being invalidate because project f15ce62009c64053a42a3d1f9d7e7630 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:24:31.239909 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-d61afedf-e9eb-4e14-8dd3-71a55f6417b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:31.240565 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-d61afedf-e9eb-4e14-8dd3-71a55f6417b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:31.242122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1276/2544] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:24:31 2022] DELETE /identity/v3/projects/f15ce62009c64053a42a3d1f9d7e7630 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:31.247452 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9405ac-f944-49c6-841b-f8b37413b98d None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:31.247739 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9405ac-f944-49c6-841b-f8b37413b98d None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:31.248014 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9405ac-f944-49c6-841b-f8b37413b98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:31.268179 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-5c9405ac-f944-49c6-841b-f8b37413b98d None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:31.269422 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5c9405ac-f944-49c6-841b-f8b37413b98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:31.328035 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1269/2545] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:24:31.336585 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa32058-dd64-43e2-b36c-1ffa992f0f4c None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:31.336876 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa32058-dd64-43e2-b36c-1ffa992f0f4c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:31.337153 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa32058-dd64-43e2-b36c-1ffa992f0f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:31.356997 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-3fa32058-dd64-43e2-b36c-1ffa992f0f4c None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:31.358184 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-3fa32058-dd64-43e2-b36c-1ffa992f0f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:31.365929 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1277/2546] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:24:31.370338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9405ac-f944-49c6-841b-f8b37413b98d None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:31.376599 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:31.380357 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9405ac-f944-49c6-841b-f8b37413b98d None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:31.386791 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af696373-9895-468d-9edd-d19acf269720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:31.387763 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af696373-9895-468d-9edd-d19acf269720 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:31.388099 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af696373-9895-468d-9edd-d19acf269720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:31.388550 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af696373-9895-468d-9edd-d19acf269720 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:31.389428 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af696373-9895-468d-9edd-d19acf269720 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:24:31.389971 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af696373-9895-468d-9edd-d19acf269720 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:31.391821 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af696373-9895-468d-9edd-d19acf269720 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:31.403992 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1270/2547] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:24:31.408251 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa32058-dd64-43e2-b36c-1ffa992f0f4c None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:31.417636 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa32058-dd64-43e2-b36c-1ffa992f0f4c None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:31.422471 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1aade6-aa0a-41d3-a994-4c01edd7a6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:31.423057 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1aade6-aa0a-41d3-a994-4c01edd7a6a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:31.423338 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1aade6-aa0a-41d3-a994-4c01edd7a6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:31.423614 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1aade6-aa0a-41d3-a994-4c01edd7a6a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:31.424235 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1aade6-aa0a-41d3-a994-4c01edd7a6a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:31.426144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1aade6-aa0a-41d3-a994-4c01edd7a6a2 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:31.454088 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1278/2548] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:24:31 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:24:31.458264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af696373-9895-468d-9edd-d19acf269720 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:31.468821 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af696373-9895-468d-9edd-d19acf269720 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:31.525096 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d4f03-dbcc-4130-a62b-312e0e9756bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:31.525689 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d4f03-dbcc-4130-a62b-312e0e9756bf admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:31.525969 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d4f03-dbcc-4130-a62b-312e0e9756bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:31.526245 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d4f03-dbcc-4130-a62b-312e0e9756bf admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:31.526867 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d4f03-dbcc-4130-a62b-312e0e9756bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:31.528774 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d4f03-dbcc-4130-a62b-312e0e9756bf admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:31.548463 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1271/2549] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:24:31 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:24:31.552685 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1aade6-aa0a-41d3-a994-4c01edd7a6a2 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:31.562729 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1aade6-aa0a-41d3-a994-4c01edd7a6a2 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:31.567371 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6bd83-0473-46ff-a902-961e489b9a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:31.567960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6bd83-0473-46ff-a902-961e489b9a54 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:31.568246 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6bd83-0473-46ff-a902-961e489b9a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:31.568539 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6bd83-0473-46ff-a902-961e489b9a54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:31.569172 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a6bd83-0473-46ff-a902-961e489b9a54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:31.571317 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a6bd83-0473-46ff-a902-961e489b9a54 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:31.580921 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1279/2550] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:24:31.584682 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d4f03-dbcc-4130-a62b-312e0e9756bf admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:31.595247 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d4f03-dbcc-4130-a62b-312e0e9756bf admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:31.600136 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d82a8b-392e-479e-a1cc-e61bd4a9e862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:31.600776 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d82a8b-392e-479e-a1cc-e61bd4a9e862 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:31.601045 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d82a8b-392e-479e-a1cc-e61bd4a9e862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:31.601325 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d82a8b-392e-479e-a1cc-e61bd4a9e862 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:31.601961 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d82a8b-392e-479e-a1cc-e61bd4a9e862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:31.603541 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d82a8b-392e-479e-a1cc-e61bd4a9e862 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:31.614413 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1272/2551] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:24:31.618456 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6bd83-0473-46ff-a902-961e489b9a54 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:31.628569 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a6bd83-0473-46ff-a902-961e489b9a54 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:31.633847 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c7d23-7cff-4526-bb50-4c000a554367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:31.634489 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c7d23-7cff-4526-bb50-4c000a554367 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:31.634770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c7d23-7cff-4526-bb50-4c000a554367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:31.635050 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c7d23-7cff-4526-bb50-4c000a554367 admin admin] PATH_INFO: `/v3/projects/99afe997a667431998cf3cab3101d22f/users/8b9f0236a2514149af7f36da7749e4d4/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:31.637638 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:31.653385 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c7d23-7cff-4526-bb50-4c000a554367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99afe997a667431998cf3cab3101d22f, user_id=8b9f0236a2514149af7f36da7749e4d4, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:31.656048 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c7d23-7cff-4526-bb50-4c000a554367 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:31.668934 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1280/2552] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:24:31 2022] PUT /identity/v3/projects/99afe997a667431998cf3cab3101d22f/users/8b9f0236a2514149af7f36da7749e4d4/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:31.680777 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e6ccf-d8d7-46d4-99af-9c712d01e7c8 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:31.681253 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e6ccf-d8d7-46d4-99af-9c712d01e7c8 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:31.681374 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e6ccf-d8d7-46d4-99af-9c712d01e7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:31.709121 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-5a8e6ccf-d8d7-46d4-99af-9c712d01e7c8 None None] MFA Rules not processed for user `8b9f0236a2514149af7f36da7749e4d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:31.710475 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-5a8e6ccf-d8d7-46d4-99af-9c712d01e7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:31.774036 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1273/2553] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:24:31.778535 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c7d23-7cff-4526-bb50-4c000a554367 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:31.839898 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c7d23-7cff-4526-bb50-4c000a554367 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:31.845101 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78671fce-b3d7-4715-a44c-22f1be25cb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:31.845710 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78671fce-b3d7-4715-a44c-22f1be25cb8b admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:31.845997 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78671fce-b3d7-4715-a44c-22f1be25cb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:31.846277 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78671fce-b3d7-4715-a44c-22f1be25cb8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:31.846922 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78671fce-b3d7-4715-a44c-22f1be25cb8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:31.849215 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78671fce-b3d7-4715-a44c-22f1be25cb8b admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:31.877221 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1281/2554] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:24:31 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:24:31.881234 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e6ccf-d8d7-46d4-99af-9c712d01e7c8 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:31.892355 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e6ccf-d8d7-46d4-99af-9c712d01e7c8 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:31.948811 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b7a19-c519-4d5c-a896-65bc06bdce21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:31.949337 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b7a19-c519-4d5c-a896-65bc06bdce21 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:31.949618 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b7a19-c519-4d5c-a896-65bc06bdce21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:31.949894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b7a19-c519-4d5c-a896-65bc06bdce21 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:31.950533 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b7a19-c519-4d5c-a896-65bc06bdce21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:31.952429 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b7a19-c519-4d5c-a896-65bc06bdce21 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:31.960449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:31.974916 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1274/2555] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:24:31 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:24:31.979144 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78671fce-b3d7-4715-a44c-22f1be25cb8b admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:31.989293 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78671fce-b3d7-4715-a44c-22f1be25cb8b admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:31.994451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376c331-9656-462e-9ffe-52b2b1f4dc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:31.995042 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376c331-9656-462e-9ffe-52b2b1f4dc49 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:31.995326 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376c331-9656-462e-9ffe-52b2b1f4dc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:31.995604 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376c331-9656-462e-9ffe-52b2b1f4dc49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:31.996239 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376c331-9656-462e-9ffe-52b2b1f4dc49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:31.998145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376c331-9656-462e-9ffe-52b2b1f4dc49 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:32.009194 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1282/2556] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:24:32.013182 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b7a19-c519-4d5c-a896-65bc06bdce21 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:32.023320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b7a19-c519-4d5c-a896-65bc06bdce21 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:32.027879 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e2a30-2170-47ca-a0bf-a7400da803a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:32.028592 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e2a30-2170-47ca-a0bf-a7400da803a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:32.028956 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e2a30-2170-47ca-a0bf-a7400da803a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:32.029188 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e2a30-2170-47ca-a0bf-a7400da803a8 admin admin] PATH_INFO: `/v3/projects/05781a7b433d459491c2c532de150b32/users/13a58afb01fa41129bdbd29f01cad124/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:32.044473 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e2a30-2170-47ca-a0bf-a7400da803a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05781a7b433d459491c2c532de150b32, user_id=13a58afb01fa41129bdbd29f01cad124, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:32.047091 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e2a30-2170-47ca-a0bf-a7400da803a8 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:32.059810 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1275/2557] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:24:32 2022] PUT /identity/v3/projects/05781a7b433d459491c2c532de150b32/users/13a58afb01fa41129bdbd29f01cad124/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:32.063683 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376c331-9656-462e-9ffe-52b2b1f4dc49 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:32.074127 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376c331-9656-462e-9ffe-52b2b1f4dc49 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:32.133100 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35b8c7-f660-45b6-a7a7-586969d05485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:32.133721 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35b8c7-f660-45b6-a7a7-586969d05485 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:32.134005 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35b8c7-f660-45b6-a7a7-586969d05485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:32.134285 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35b8c7-f660-45b6-a7a7-586969d05485 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:32.134922 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35b8c7-f660-45b6-a7a7-586969d05485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:32.136519 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35b8c7-f660-45b6-a7a7-586969d05485 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:32.146236 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1283/2558] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:24:32.150563 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e2a30-2170-47ca-a0bf-a7400da803a8 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:32.160713 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e2a30-2170-47ca-a0bf-a7400da803a8 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:32.166202 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c18d3b-2c49-4461-8966-46b540f4dea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:32.166841 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c18d3b-2c49-4461-8966-46b540f4dea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:32.167120 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c18d3b-2c49-4461-8966-46b540f4dea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:32.167392 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c18d3b-2c49-4461-8966-46b540f4dea5 admin admin] PATH_INFO: `/v3/projects/05781a7b433d459491c2c532de150b32/users/13a58afb01fa41129bdbd29f01cad124/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:32.169326 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c18d3b-2c49-4461-8966-46b540f4dea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05781a7b433d459491c2c532de150b32, user_id=13a58afb01fa41129bdbd29f01cad124, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:32.171910 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c18d3b-2c49-4461-8966-46b540f4dea5 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:32.183226 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1276/2559] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:24:32 2022] PUT /identity/v3/projects/05781a7b433d459491c2c532de150b32/users/13a58afb01fa41129bdbd29f01cad124/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:32.194501 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51951c90-bf89-45e1-a7e5-82c6bf94e05b None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:32.195346 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51951c90-bf89-45e1-a7e5-82c6bf94e05b None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:32.195631 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51951c90-bf89-45e1-a7e5-82c6bf94e05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:32.224300 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-51951c90-bf89-45e1-a7e5-82c6bf94e05b None None] MFA Rules not processed for user `13a58afb01fa41129bdbd29f01cad124`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:32.225531 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-51951c90-bf89-45e1-a7e5-82c6bf94e05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:32.267844 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:32.296763 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1284/2560] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:32 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:24:32.307992 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c18d3b-2c49-4461-8966-46b540f4dea5 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:32.309815 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-c2c18d3b-2c49-4461-8966-46b540f4dea5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:32.368732 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c18d3b-2c49-4461-8966-46b540f4dea5 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:32.372737 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2149f8a-ff2f-4d32-a9c0-927aa2200233 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:32.373338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2149f8a-ff2f-4d32-a9c0-927aa2200233 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:32.373617 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2149f8a-ff2f-4d32-a9c0-927aa2200233 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:32.373888 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2149f8a-ff2f-4d32-a9c0-927aa2200233 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:32.383148 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2149f8a-ff2f-4d32-a9c0-927aa2200233 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:32.384595 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2149f8a-ff2f-4d32-a9c0-927aa2200233 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:32.393857 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1277/2561] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:24:32 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:24:32.495692 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51951c90-bf89-45e1-a7e5-82c6bf94e05b None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:32.505440 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51951c90-bf89-45e1-a7e5-82c6bf94e05b None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:32.509441 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585f772-bbe7-4a34-84b7-30db0582ade6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:32.510033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585f772-bbe7-4a34-84b7-30db0582ade6 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:32.510311 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585f772-bbe7-4a34-84b7-30db0582ade6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:32.510584 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585f772-bbe7-4a34-84b7-30db0582ade6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:32.566812 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585f772-bbe7-4a34-84b7-30db0582ade6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:32.568249 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585f772-bbe7-4a34-84b7-30db0582ade6 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:32.577311 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1285/2562] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:24:32 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:24:35.175633 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2149f8a-ff2f-4d32-a9c0-927aa2200233 service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:35.238654 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2149f8a-ff2f-4d32-a9c0-927aa2200233 service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:35.244362 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac711db3-1a07-456c-a07a-538658b65efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:35.245013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac711db3-1a07-456c-a07a-538658b65efd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:35.245320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac711db3-1a07-456c-a07a-538658b65efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:35.245574 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac711db3-1a07-456c-a07a-538658b65efd admin admin] PATH_INFO: `/v3/users/8b9f0236a2514149af7f36da7749e4d4` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:35.246667 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac711db3-1a07-456c-a07a-538658b65efd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b9f0236a2514149af7f36da7749e4d4)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:35.251793 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac711db3-1a07-456c-a07a-538658b65efd admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:35.303894 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-ac711db3-1a07-456c-a07a-538658b65efd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b9f0236a2514149af7f36da7749e4d4'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:35.310849 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-ac711db3-1a07-456c-a07a-538658b65efd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b9f0236a2514149af7f36da7749e4d4'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:35.311952 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:35.325390 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-ac711db3-1a07-456c-a07a-538658b65efd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b9f0236a2514149af7f36da7749e4d4'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:35.326068 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-ac711db3-1a07-456c-a07a-538658b65efd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b9f0236a2514149af7f36da7749e4d4'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:35.326679 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-ac711db3-1a07-456c-a07a-538658b65efd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b9f0236a2514149af7f36da7749e4d4'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:35.340032 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1278/2563] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:24:35 2022] DELETE /identity/v3/users/8b9f0236a2514149af7f36da7749e4d4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:35.344718 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d47e6b-91a3-4d1b-8a6e-e05b6871b518 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:35.345226 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d47e6b-91a3-4d1b-8a6e-e05b6871b518 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:35.345342 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d47e6b-91a3-4d1b-8a6e-e05b6871b518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:35.364548 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-13d47e6b-91a3-4d1b-8a6e-e05b6871b518 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:35.365645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-13d47e6b-91a3-4d1b-8a6e-e05b6871b518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:35.428373 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1286/2564] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:24:35.438797 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac711db3-1a07-456c-a07a-538658b65efd admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:35.440235 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ac711db3-1a07-456c-a07a-538658b65efd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:35.501243 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac711db3-1a07-456c-a07a-538658b65efd admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:35.504998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156dcb2-3663-44d8-a123-073462d563b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:35.505607 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156dcb2-3663-44d8-a123-073462d563b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:35.505887 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156dcb2-3663-44d8-a123-073462d563b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:35.506166 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156dcb2-3663-44d8-a123-073462d563b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:35.518181 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156dcb2-3663-44d8-a123-073462d563b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:35.519861 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156dcb2-3663-44d8-a123-073462d563b3 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:35.534191 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1279/2565] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:24:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:24:35.828771 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d47e6b-91a3-4d1b-8a6e-e05b6871b518 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:35.830259 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-13d47e6b-91a3-4d1b-8a6e-e05b6871b518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:35.844204 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d47e6b-91a3-4d1b-8a6e-e05b6871b518 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:35.849373 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b71a1a-7783-4005-aed4-14a669f1e60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:35.850083 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b71a1a-7783-4005-aed4-14a669f1e60c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:35.850434 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b71a1a-7783-4005-aed4-14a669f1e60c admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:35.851322 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b71a1a-7783-4005-aed4-14a669f1e60c admin admin] PATH_INFO: `/v3/projects/99afe997a667431998cf3cab3101d22f` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:35.852719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b71a1a-7783-4005-aed4-14a669f1e60c admin admin] RBAC: Authorizing `identity:delete_project(project_id=99afe997a667431998cf3cab3101d22f)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:35.855028 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b71a1a-7783-4005-aed4-14a669f1e60c admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:35.902978 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-05b71a1a-7783-4005-aed4-14a669f1e60c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99afe997a667431998cf3cab3101d22f', 'request_id': 'req-05b71a1a-7783-4005-aed4-14a669f1e60c'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:35.910130 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-05b71a1a-7783-4005-aed4-14a669f1e60c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99afe997a667431998cf3cab3101d22f', 'request_id': 'req-05b71a1a-7783-4005-aed4-14a669f1e60c'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:35.916713 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-05b71a1a-7783-4005-aed4-14a669f1e60c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99afe997a667431998cf3cab3101d22f', 'request_id': 'req-05b71a1a-7783-4005-aed4-14a669f1e60c'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:35.924257 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:35.928294 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-05b71a1a-7783-4005-aed4-14a669f1e60c admin admin] The token cache is being invalidate because project 99afe997a667431998cf3cab3101d22f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:24:35.929520 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-05b71a1a-7783-4005-aed4-14a669f1e60c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:35.930503 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-05b71a1a-7783-4005-aed4-14a669f1e60c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:35.932465 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1287/2566] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:24:35 2022] DELETE /identity/v3/projects/99afe997a667431998cf3cab3101d22f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:35.936533 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156dcb2-3663-44d8-a123-073462d563b3 service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:35.946570 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156dcb2-3663-44d8-a123-073462d563b3 service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:36.003928 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c565e-078c-4ea0-b3cb-fe80b2bb9b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:36.004588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c565e-078c-4ea0-b3cb-fe80b2bb9b1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:36.004877 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c565e-078c-4ea0-b3cb-fe80b2bb9b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:36.005154 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c565e-078c-4ea0-b3cb-fe80b2bb9b1f admin admin] PATH_INFO: `/v3/users/13a58afb01fa41129bdbd29f01cad124` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:36.006291 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c565e-078c-4ea0-b3cb-fe80b2bb9b1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=13a58afb01fa41129bdbd29f01cad124)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:36.008251 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c565e-078c-4ea0-b3cb-fe80b2bb9b1f admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:36.055043 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1d9c565e-078c-4ea0-b3cb-fe80b2bb9b1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13a58afb01fa41129bdbd29f01cad124'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:36.061957 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1d9c565e-078c-4ea0-b3cb-fe80b2bb9b1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13a58afb01fa41129bdbd29f01cad124'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:36.073918 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1d9c565e-078c-4ea0-b3cb-fe80b2bb9b1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13a58afb01fa41129bdbd29f01cad124'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:36.074587 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1d9c565e-078c-4ea0-b3cb-fe80b2bb9b1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13a58afb01fa41129bdbd29f01cad124'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:36.075198 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-1d9c565e-078c-4ea0-b3cb-fe80b2bb9b1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13a58afb01fa41129bdbd29f01cad124'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:36.078779 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:36.090710 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1280/2567] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:24:35 2022] DELETE /identity/v3/users/13a58afb01fa41129bdbd29f01cad124 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:36.374456 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b71a1a-7783-4005-aed4-14a669f1e60c admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:36.375957 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-05b71a1a-7783-4005-aed4-14a669f1e60c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:36.393764 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b71a1a-7783-4005-aed4-14a669f1e60c admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:36.450719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d98f46-fe0d-4153-919d-2b7384c4ff79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:36.451272 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d98f46-fe0d-4153-919d-2b7384c4ff79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:36.451557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d98f46-fe0d-4153-919d-2b7384c4ff79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:36.451835 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d98f46-fe0d-4153-919d-2b7384c4ff79 admin admin] PATH_INFO: `/v3/projects/05781a7b433d459491c2c532de150b32` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:36.452984 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d98f46-fe0d-4153-919d-2b7384c4ff79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05781a7b433d459491c2c532de150b32)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:36.454902 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d98f46-fe0d-4153-919d-2b7384c4ff79 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:36.502487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-14d98f46-fe0d-4153-919d-2b7384c4ff79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05781a7b433d459491c2c532de150b32', 'request_id': 'req-14d98f46-fe0d-4153-919d-2b7384c4ff79'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:36.510193 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-14d98f46-fe0d-4153-919d-2b7384c4ff79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05781a7b433d459491c2c532de150b32', 'request_id': 'req-14d98f46-fe0d-4153-919d-2b7384c4ff79'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:36.515724 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-14d98f46-fe0d-4153-919d-2b7384c4ff79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05781a7b433d459491c2c532de150b32', 'request_id': 'req-14d98f46-fe0d-4153-919d-2b7384c4ff79'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:36.527405 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-14d98f46-fe0d-4153-919d-2b7384c4ff79 admin admin] The token cache is being invalidate because project 05781a7b433d459491c2c532de150b32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:24:36.527693 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-14d98f46-fe0d-4153-919d-2b7384c4ff79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:36.528327 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-14d98f46-fe0d-4153-919d-2b7384c4ff79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:36.529862 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1288/2568] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:24:36 2022] DELETE /identity/v3/projects/05781a7b433d459491c2c532de150b32 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:36.536055 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a143b17-0ff1-4058-86d2-c72cbec2e4de None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:36.536568 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a143b17-0ff1-4058-86d2-c72cbec2e4de None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:36.536702 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a143b17-0ff1-4058-86d2-c72cbec2e4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:36.555923 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-0a143b17-0ff1-4058-86d2-c72cbec2e4de None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:36.557154 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-0a143b17-0ff1-4058-86d2-c72cbec2e4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:36.614980 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1281/2569] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:24:36.624412 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01515b-9b08-4fd8-b48b-004b7f2f14c1 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:36.624857 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01515b-9b08-4fd8-b48b-004b7f2f14c1 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:36.625148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01515b-9b08-4fd8-b48b-004b7f2f14c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:36.645310 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-fb01515b-9b08-4fd8-b48b-004b7f2f14c1 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:36.646525 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-fb01515b-9b08-4fd8-b48b-004b7f2f14c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:36.655430 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1289/2570] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:24:36.659645 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a143b17-0ff1-4058-86d2-c72cbec2e4de None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:36.669521 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a143b17-0ff1-4058-86d2-c72cbec2e4de None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:36.674201 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0b305-0738-407b-8de8-97c64cccee63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:36.674811 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0b305-0738-407b-8de8-97c64cccee63 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:36.675097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0b305-0738-407b-8de8-97c64cccee63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:36.675376 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b0b305-0738-407b-8de8-97c64cccee63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:36.676034 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b0b305-0738-407b-8de8-97c64cccee63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:24:36.676344 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b0b305-0738-407b-8de8-97c64cccee63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:36.678000 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b0b305-0738-407b-8de8-97c64cccee63 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:36.689715 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1282/2571] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:24:36.694172 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01515b-9b08-4fd8-b48b-004b7f2f14c1 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:36.704132 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01515b-9b08-4fd8-b48b-004b7f2f14c1 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:36.708807 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3799b-5867-4aef-9335-d06e2d897a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:36.709425 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3799b-5867-4aef-9335-d06e2d897a51 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:36.709732 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3799b-5867-4aef-9335-d06e2d897a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:36.710010 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3799b-5867-4aef-9335-d06e2d897a51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:36.711322 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3799b-5867-4aef-9335-d06e2d897a51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:36.713242 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea3799b-5867-4aef-9335-d06e2d897a51 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:36.740034 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1290/2572] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:24:36 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:24:36.744325 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0b305-0738-407b-8de8-97c64cccee63 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:36.754579 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b0b305-0738-407b-8de8-97c64cccee63 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:36.810554 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb7c3d-8d93-4695-95a4-d3a061e14eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:36.811157 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb7c3d-8d93-4695-95a4-d3a061e14eac admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:36.811442 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb7c3d-8d93-4695-95a4-d3a061e14eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:36.811720 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb7c3d-8d93-4695-95a4-d3a061e14eac admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:36.812356 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb7c3d-8d93-4695-95a4-d3a061e14eac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:36.814292 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb7c3d-8d93-4695-95a4-d3a061e14eac admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:36.839871 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1283/2573] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:24:36 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:24:36.841026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3799b-5867-4aef-9335-d06e2d897a51 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:36.851533 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3799b-5867-4aef-9335-d06e2d897a51 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:36.857729 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297936a0-db97-4a57-96f0-ec96f5466ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:36.858358 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297936a0-db97-4a57-96f0-ec96f5466ab1 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:36.858641 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297936a0-db97-4a57-96f0-ec96f5466ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:36.858919 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297936a0-db97-4a57-96f0-ec96f5466ab1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:36.859560 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297936a0-db97-4a57-96f0-ec96f5466ab1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:36.861193 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297936a0-db97-4a57-96f0-ec96f5466ab1 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:36.871556 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1291/2574] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:24:36.875706 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb7c3d-8d93-4695-95a4-d3a061e14eac admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:36.886772 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb7c3d-8d93-4695-95a4-d3a061e14eac admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:36.891346 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae087676-9fab-4862-902f-02c69bb38593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:36.891943 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae087676-9fab-4862-902f-02c69bb38593 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:36.892229 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae087676-9fab-4862-902f-02c69bb38593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:36.892539 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae087676-9fab-4862-902f-02c69bb38593 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:36.893209 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae087676-9fab-4862-902f-02c69bb38593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:36.894794 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae087676-9fab-4862-902f-02c69bb38593 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:36.906075 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1284/2575] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:24:36.910093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297936a0-db97-4a57-96f0-ec96f5466ab1 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:36.920155 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297936a0-db97-4a57-96f0-ec96f5466ab1 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:36.924815 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1478e5ea-809a-4643-a1ab-07f83c618507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:36.925456 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1478e5ea-809a-4643-a1ab-07f83c618507 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:36.925741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1478e5ea-809a-4643-a1ab-07f83c618507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:36.926018 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1478e5ea-809a-4643-a1ab-07f83c618507 admin admin] PATH_INFO: `/v3/projects/bdc9747f00794db98384c7a51d2d98e1/users/c380b0ec2d52401e8dc89f957ece313f/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:36.941454 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1478e5ea-809a-4643-a1ab-07f83c618507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc9747f00794db98384c7a51d2d98e1, user_id=c380b0ec2d52401e8dc89f957ece313f, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:36.944215 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1478e5ea-809a-4643-a1ab-07f83c618507 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:36.958325 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1292/2576] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:24:36 2022] PUT /identity/v3/projects/bdc9747f00794db98384c7a51d2d98e1/users/c380b0ec2d52401e8dc89f957ece313f/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:36.968565 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd595d-b68f-4cf7-a2f9-f6ccd666d632 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:36.968858 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd595d-b68f-4cf7-a2f9-f6ccd666d632 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:36.969135 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd595d-b68f-4cf7-a2f9-f6ccd666d632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:36.996776 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-44cd595d-b68f-4cf7-a2f9-f6ccd666d632 None None] MFA Rules not processed for user `c380b0ec2d52401e8dc89f957ece313f`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:36.998223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-44cd595d-b68f-4cf7-a2f9-f6ccd666d632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:37.047313 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:37.065526 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1285/2577] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:24:37.076928 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1478e5ea-809a-4643-a1ab-07f83c618507 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:37.078622 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-1478e5ea-809a-4643-a1ab-07f83c618507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:37.113968 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:37.143465 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1478e5ea-809a-4643-a1ab-07f83c618507 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:37.147084 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609ccd6-994a-4b5d-88dc-9197404f8a19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:37.147786 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609ccd6-994a-4b5d-88dc-9197404f8a19 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:37.148256 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609ccd6-994a-4b5d-88dc-9197404f8a19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:37.148568 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609ccd6-994a-4b5d-88dc-9197404f8a19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:37.156608 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609ccd6-994a-4b5d-88dc-9197404f8a19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:37.158012 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609ccd6-994a-4b5d-88dc-9197404f8a19 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:37.166749 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1293/2578] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:24:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:24:38.524023 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40a17f-52d9-4791-a4e3-da6dd7987604 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:38.524551 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40a17f-52d9-4791-a4e3-da6dd7987604 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:38.524674 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40a17f-52d9-4791-a4e3-da6dd7987604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:38.543824 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-4a40a17f-52d9-4791-a4e3-da6dd7987604 None None] MFA Rules not processed for user `c380b0ec2d52401e8dc89f957ece313f`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:38.545042 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-4a40a17f-52d9-4791-a4e3-da6dd7987604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:38.551786 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1286/2579] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:24:40.826669 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609ccd6-994a-4b5d-88dc-9197404f8a19 service cinder] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:40.828139 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9609ccd6-994a-4b5d-88dc-9197404f8a19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:40.898556 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609ccd6-994a-4b5d-88dc-9197404f8a19 service cinder] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:40.902840 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f51caf-b3a3-40f9-9ed3-5f67fb15182b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:40.903451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f51caf-b3a3-40f9-9ed3-5f67fb15182b service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:40.903734 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f51caf-b3a3-40f9-9ed3-5f67fb15182b service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:40.904009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f51caf-b3a3-40f9-9ed3-5f67fb15182b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:40.905784 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-24f51caf-b3a3-40f9-9ed3-5f67fb15182b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:40.916717 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f51caf-b3a3-40f9-9ed3-5f67fb15182b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:40.918160 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f51caf-b3a3-40f9-9ed3-5f67fb15182b service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:40.926218 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1294/2580] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:24:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:24:48.538468 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=7 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:50.922419 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=40 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:52.189125 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257c841-9051-4774-b249-df9470ec08ad None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:52.189638 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257c841-9051-4774-b249-df9470ec08ad None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:52.189741 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9257c841-9051-4774-b249-df9470ec08ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:52.237846 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-9257c841-9051-4774-b249-df9470ec08ad None None] MFA Rules not processed for user `7c0356072f92451f98b03a2225e22f86`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:52.239039 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-9257c841-9051-4774-b249-df9470ec08ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:52.301475 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1287/2581] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 00:24:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:24:52.346301 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f51caf-b3a3-40f9-9ed3-5f67fb15182b service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:52.347770 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-24f51caf-b3a3-40f9-9ed3-5f67fb15182b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:52.438019 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f51caf-b3a3-40f9-9ed3-5f67fb15182b service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:52.443135 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1e6fe-5173-4bb0-8726-2f9a59480b0b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfd8c7516b7c435d88298672b2596d6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:52.443732 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1e6fe-5173-4bb0-8726-2f9a59480b0b service swift] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:52.444011 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1e6fe-5173-4bb0-8726-2f9a59480b0b service swift] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:52.444285 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c1e6fe-5173-4bb0-8726-2f9a59480b0b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:52.452612 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1e6fe-5173-4bb0-8726-2f9a59480b0b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:52.454192 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c1e6fe-5173-4bb0-8726-2f9a59480b0b service swift] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:52.462054 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1295/2582] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 00:24:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:24:52.956814 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e0c23-1469-4672-8e09-4b8c6c827a0d None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:52.957108 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e0c23-1469-4672-8e09-4b8c6c827a0d None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:52.957391 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e0c23-1469-4672-8e09-4b8c6c827a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:52.978234 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-bb7e0c23-1469-4672-8e09-4b8c6c827a0d None None] MFA Rules not processed for user `7c0356072f92451f98b03a2225e22f86`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:52.979423 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-bb7e0c23-1469-4672-8e09-4b8c6c827a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:52.986509 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1288/2583] 10.222.0.13 () {58 vars in 967 bytes} [Sun Oct 23 00:24:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:24:52.998695 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1e6fe-5173-4bb0-8726-2f9a59480b0b service swift] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:53.009944 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c1e6fe-5173-4bb0-8726-2f9a59480b0b service swift] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:53.014940 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f922f-3e5f-4a3f-aa3a-9a5a051cb7a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfd8c7516b7c435d88298672b2596d6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:53.015550 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f922f-3e5f-4a3f-aa3a-9a5a051cb7a7 service swift] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:53.015818 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f922f-3e5f-4a3f-aa3a-9a5a051cb7a7 service swift] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:53.016095 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520f922f-3e5f-4a3f-aa3a-9a5a051cb7a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:53.024729 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f922f-3e5f-4a3f-aa3a-9a5a051cb7a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:53.026290 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520f922f-3e5f-4a3f-aa3a-9a5a051cb7a7 service swift] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:53.034154 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1296/2584] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 00:24:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:24:53.256721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8ee29-1fd1-4af8-ad14-221b793a3c73 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:53.257195 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8ee29-1fd1-4af8-ad14-221b793a3c73 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:53.257309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8ee29-1fd1-4af8-ad14-221b793a3c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:53.277257 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-16c8ee29-1fd1-4af8-ad14-221b793a3c73 None None] MFA Rules not processed for user `7c0356072f92451f98b03a2225e22f86`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:53.278461 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-16c8ee29-1fd1-4af8-ad14-221b793a3c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:53.285588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1289/2585] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 00:24:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:24:53.297306 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f922f-3e5f-4a3f-aa3a-9a5a051cb7a7 service swift] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:53.307608 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520f922f-3e5f-4a3f-aa3a-9a5a051cb7a7 service swift] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:53.312668 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd5988-91f5-4f0e-9e47-e604fdb267fd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfd8c7516b7c435d88298672b2596d6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:53.313281 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd5988-91f5-4f0e-9e47-e604fdb267fd service swift] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:53.313610 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd5988-91f5-4f0e-9e47-e604fdb267fd service swift] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:53.313835 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd5988-91f5-4f0e-9e47-e604fdb267fd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:53.322494 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd5988-91f5-4f0e-9e47-e604fdb267fd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:53.324068 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd5988-91f5-4f0e-9e47-e604fdb267fd service swift] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:53.332151 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1297/2586] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 00:24:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:24:53.651731 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9e14a-996b-4429-a069-2ad38b2935fd None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:53.652022 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9e14a-996b-4429-a069-2ad38b2935fd None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:53.652303 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9e14a-996b-4429-a069-2ad38b2935fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:53.671253 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-92f9e14a-996b-4429-a069-2ad38b2935fd None None] MFA Rules not processed for user `7c0356072f92451f98b03a2225e22f86`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:53.672444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-92f9e14a-996b-4429-a069-2ad38b2935fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:53.679456 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1290/2587] 10.222.0.13 () {58 vars in 967 bytes} [Sun Oct 23 00:24:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:24:53.692072 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd5988-91f5-4f0e-9e47-e604fdb267fd service swift] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:53.703360 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd5988-91f5-4f0e-9e47-e604fdb267fd service swift] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:53.708413 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced4c64-8d2b-4a14-90d0-2edc179458d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfd8c7516b7c435d88298672b2596d6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:53.709029 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced4c64-8d2b-4a14-90d0-2edc179458d1 service swift] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:53.709309 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced4c64-8d2b-4a14-90d0-2edc179458d1 service swift] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:53.709581 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced4c64-8d2b-4a14-90d0-2edc179458d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:53.717508 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced4c64-8d2b-4a14-90d0-2edc179458d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:53.719068 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced4c64-8d2b-4a14-90d0-2edc179458d1 service swift] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:53.727445 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1298/2588] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 00:24:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:24:55.986985 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9e14a-996b-4429-a069-2ad38b2935fd None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:56.054297 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9e14a-996b-4429-a069-2ad38b2935fd None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:56.059622 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bbac4-fa9f-4b69-92c5-20b696da38a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:56.060256 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bbac4-fa9f-4b69-92c5-20b696da38a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:56.060584 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bbac4-fa9f-4b69-92c5-20b696da38a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:56.060864 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bbac4-fa9f-4b69-92c5-20b696da38a2 admin admin] PATH_INFO: `/v3/users/c380b0ec2d52401e8dc89f957ece313f` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:56.062037 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bbac4-fa9f-4b69-92c5-20b696da38a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c380b0ec2d52401e8dc89f957ece313f)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:56.063948 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bbac4-fa9f-4b69-92c5-20b696da38a2 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:56.117986 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-b65bbac4-fa9f-4b69-92c5-20b696da38a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c380b0ec2d52401e8dc89f957ece313f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:56.125339 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:56.127805 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-b65bbac4-fa9f-4b69-92c5-20b696da38a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c380b0ec2d52401e8dc89f957ece313f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:56.141324 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-b65bbac4-fa9f-4b69-92c5-20b696da38a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c380b0ec2d52401e8dc89f957ece313f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:56.142066 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-b65bbac4-fa9f-4b69-92c5-20b696da38a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c380b0ec2d52401e8dc89f957ece313f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:56.142733 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-b65bbac4-fa9f-4b69-92c5-20b696da38a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c380b0ec2d52401e8dc89f957ece313f'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:56.158340 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1291/2589] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:24:55 2022] DELETE /identity/v3/users/c380b0ec2d52401e8dc89f957ece313f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:56.163128 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711729d-bf22-4820-95d6-6ac0db783bb6 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:56.163643 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711729d-bf22-4820-95d6-6ac0db783bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:56.163749 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711729d-bf22-4820-95d6-6ac0db783bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:56.184420 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-9711729d-bf22-4820-95d6-6ac0db783bb6 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:56.185669 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9711729d-bf22-4820-95d6-6ac0db783bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:56.246225 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1299/2590] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:24:56.257022 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bbac4-fa9f-4b69-92c5-20b696da38a2 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:56.258488 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-b65bbac4-fa9f-4b69-92c5-20b696da38a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:56.323827 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bbac4-fa9f-4b69-92c5-20b696da38a2 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:56.328250 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccca48f3-861d-4ba7-9ce9-bfda6fc837fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:56.329223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca48f3-861d-4ba7-9ce9-bfda6fc837fd service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:56.329591 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca48f3-861d-4ba7-9ce9-bfda6fc837fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:56.330065 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccca48f3-861d-4ba7-9ce9-bfda6fc837fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:56.343555 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca48f3-861d-4ba7-9ce9-bfda6fc837fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:56.345011 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccca48f3-861d-4ba7-9ce9-bfda6fc837fd service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:56.355293 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1292/2591] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:24:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:24:56.522071 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711729d-bf22-4820-95d6-6ac0db783bb6 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:56.523494 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9711729d-bf22-4820-95d6-6ac0db783bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:56.536269 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711729d-bf22-4820-95d6-6ac0db783bb6 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:56.541266 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceac861-b052-483f-8d36-8f012603a71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:56.542847 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceac861-b052-483f-8d36-8f012603a71e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:56.543202 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceac861-b052-483f-8d36-8f012603a71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:56.543486 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceac861-b052-483f-8d36-8f012603a71e admin admin] PATH_INFO: `/v3/projects/bdc9747f00794db98384c7a51d2d98e1` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:56.544942 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceac861-b052-483f-8d36-8f012603a71e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdc9747f00794db98384c7a51d2d98e1)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:56.546941 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceac861-b052-483f-8d36-8f012603a71e admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:56.577809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:56.595443 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-2ceac861-b052-483f-8d36-8f012603a71e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdc9747f00794db98384c7a51d2d98e1', 'request_id': 'req-2ceac861-b052-483f-8d36-8f012603a71e'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:56.603005 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-2ceac861-b052-483f-8d36-8f012603a71e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdc9747f00794db98384c7a51d2d98e1', 'request_id': 'req-2ceac861-b052-483f-8d36-8f012603a71e'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:56.609444 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-2ceac861-b052-483f-8d36-8f012603a71e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdc9747f00794db98384c7a51d2d98e1', 'request_id': 'req-2ceac861-b052-483f-8d36-8f012603a71e'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:56.620307 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-2ceac861-b052-483f-8d36-8f012603a71e admin admin] The token cache is being invalidate because project bdc9747f00794db98384c7a51d2d98e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81380) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:24:56.620624 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-2ceac861-b052-483f-8d36-8f012603a71e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:56.621260 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-2ceac861-b052-483f-8d36-8f012603a71e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:24:56.623032 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1300/2592] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:24:56 2022] DELETE /identity/v3/projects/bdc9747f00794db98384c7a51d2d98e1 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:24:56.628009 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93011dc2-e976-48aa-9e01-cbc336b1159e None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:56.628301 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93011dc2-e976-48aa-9e01-cbc336b1159e None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:56.628610 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93011dc2-e976-48aa-9e01-cbc336b1159e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:56.648732 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-93011dc2-e976-48aa-9e01-cbc336b1159e None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:56.649943 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-93011dc2-e976-48aa-9e01-cbc336b1159e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:56.711444 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1293/2593] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:24:56.720318 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528b689-a6e3-4fbe-8c8e-5b6a1a90d8eb None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:56.720639 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528b689-a6e3-4fbe-8c8e-5b6a1a90d8eb None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:56.720949 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528b689-a6e3-4fbe-8c8e-5b6a1a90d8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:56.740719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-8528b689-a6e3-4fbe-8c8e-5b6a1a90d8eb None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:56.741933 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8528b689-a6e3-4fbe-8c8e-5b6a1a90d8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:56.750131 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1301/2594] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:24:56.754480 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93011dc2-e976-48aa-9e01-cbc336b1159e None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:56.765383 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93011dc2-e976-48aa-9e01-cbc336b1159e None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:56.769962 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8ff9a-d56f-45c5-a43b-9213f197a6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:56.770560 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8ff9a-d56f-45c5-a43b-9213f197a6ee admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:56.770844 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8ff9a-d56f-45c5-a43b-9213f197a6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:56.771119 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8ff9a-d56f-45c5-a43b-9213f197a6ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:56.771760 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a8ff9a-d56f-45c5-a43b-9213f197a6ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:24:56.772067 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a8ff9a-d56f-45c5-a43b-9213f197a6ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:56.773700 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a8ff9a-d56f-45c5-a43b-9213f197a6ee admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:56.786155 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1294/2595] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 00:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:24:56.790346 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528b689-a6e3-4fbe-8c8e-5b6a1a90d8eb None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:56.805411 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528b689-a6e3-4fbe-8c8e-5b6a1a90d8eb None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:56.811966 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4a162-8b23-4df7-86c9-6d9e75a136f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:56.812595 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4a162-8b23-4df7-86c9-6d9e75a136f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:56.812876 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4a162-8b23-4df7-86c9-6d9e75a136f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:56.813148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4a162-8b23-4df7-86c9-6d9e75a136f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:56.813777 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4a162-8b23-4df7-86c9-6d9e75a136f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:56.815658 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb4a162-8b23-4df7-86c9-6d9e75a136f2 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:56.850126 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1302/2596] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:24:56 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:24:56.854421 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8ff9a-d56f-45c5-a43b-9213f197a6ee admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:56.864456 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8ff9a-d56f-45c5-a43b-9213f197a6ee admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:56.920003 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d4f86-00f2-448d-ada3-9277aa3ec985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:56.920632 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d4f86-00f2-448d-ada3-9277aa3ec985 admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:56.920926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d4f86-00f2-448d-ada3-9277aa3ec985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:56.921196 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d4f86-00f2-448d-ada3-9277aa3ec985 admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:56.921826 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d4f86-00f2-448d-ada3-9277aa3ec985 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:56.923724 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d4f86-00f2-448d-ada3-9277aa3ec985 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:56.932540 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:24:56.945649 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1295/2597] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:24:56 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:24:56.950155 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4a162-8b23-4df7-86c9-6d9e75a136f2 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:56.960593 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4a162-8b23-4df7-86c9-6d9e75a136f2 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:56.965909 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059fb29-9811-444d-ad5c-3eceaf259730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:56.966511 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059fb29-9811-444d-ad5c-3eceaf259730 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:56.966827 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059fb29-9811-444d-ad5c-3eceaf259730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:56.967093 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059fb29-9811-444d-ad5c-3eceaf259730 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:56.967736 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c059fb29-9811-444d-ad5c-3eceaf259730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:56.969370 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c059fb29-9811-444d-ad5c-3eceaf259730 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:56.979694 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1303/2598] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:24:56.983971 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d4f86-00f2-448d-ada3-9277aa3ec985 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:56.993765 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d4f86-00f2-448d-ada3-9277aa3ec985 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:56.998733 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389848ac-a28f-4a6f-9e43-f1136dd45a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:56.999323 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389848ac-a28f-4a6f-9e43-f1136dd45a81 admin admin] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:56.999605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389848ac-a28f-4a6f-9e43-f1136dd45a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:56.999882 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389848ac-a28f-4a6f-9e43-f1136dd45a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:57.000533 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389848ac-a28f-4a6f-9e43-f1136dd45a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:57.002097 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389848ac-a28f-4a6f-9e43-f1136dd45a81 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:57.013106 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1296/2599] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:24:57.017259 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059fb29-9811-444d-ad5c-3eceaf259730 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:57.027645 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059fb29-9811-444d-ad5c-3eceaf259730 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:57.032284 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df49275-74f7-4d9f-9a36-ace36dba6678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:57.032956 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df49275-74f7-4d9f-9a36-ace36dba6678 admin admin] REQUEST_METHOD: `PUT` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:57.033240 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df49275-74f7-4d9f-9a36-ace36dba6678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:57.033513 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df49275-74f7-4d9f-9a36-ace36dba6678 admin admin] PATH_INFO: `/v3/projects/4a4117126a8d4004a3878e10554b0d5c/users/163f3d7a305f4d8b8eb14794a51acda4/roles/4bd2874c95634a50aac9fccbddc7c674` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:57.047758 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df49275-74f7-4d9f-9a36-ace36dba6678 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a4117126a8d4004a3878e10554b0d5c, user_id=163f3d7a305f4d8b8eb14794a51acda4, role_id=4bd2874c95634a50aac9fccbddc7c674)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:57.050425 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df49275-74f7-4d9f-9a36-ace36dba6678 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:57.064177 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1304/2600] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:24:57 2022] PUT /identity/v3/projects/4a4117126a8d4004a3878e10554b0d5c/users/163f3d7a305f4d8b8eb14794a51acda4/roles/4bd2874c95634a50aac9fccbddc7c674 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:24:57.069390 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59179955-46dd-4148-9e26-24c99781a153 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:57.069681 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59179955-46dd-4148-9e26-24c99781a153 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:57.069961 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59179955-46dd-4148-9e26-24c99781a153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:57.089759 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-59179955-46dd-4148-9e26-24c99781a153 None None] MFA Rules not processed for user `ac8d80b2c4a24dfea25c6de809bc37e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:24:57.090957 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-59179955-46dd-4148-9e26-24c99781a153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:57.150691 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1297/2601] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:24:57.160244 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df49275-74f7-4d9f-9a36-ace36dba6678 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:24:57.162024 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-2df49275-74f7-4d9f-9a36-ace36dba6678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:24:57.218377 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df49275-74f7-4d9f-9a36-ace36dba6678 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:24:57.222033 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31980e99-10d5-4dff-a92b-5dff0dd19796 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:24:57.222637 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31980e99-10d5-4dff-a92b-5dff0dd19796 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:24:57.222923 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31980e99-10d5-4dff-a92b-5dff0dd19796 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:24:57.223210 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31980e99-10d5-4dff-a92b-5dff0dd19796 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:24:57.232083 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31980e99-10d5-4dff-a92b-5dff0dd19796 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:24:57.233541 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31980e99-10d5-4dff-a92b-5dff0dd19796 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:24:57.242790 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1305/2602] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:24:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:25:01.250316 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39a385-e863-4d87-91eb-5dd4538ebe86 None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:25:01.250850 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39a385-e863-4d87-91eb-5dd4538ebe86 None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:25:01.250945 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39a385-e863-4d87-91eb-5dd4538ebe86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:25:01.293080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-2b39a385-e863-4d87-91eb-5dd4538ebe86 None None] MFA Rules not processed for user `163f3d7a305f4d8b8eb14794a51acda4`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:25:01.295885 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-2b39a385-e863-4d87-91eb-5dd4538ebe86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:25:01.384899 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1298/2603] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:25:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:25:01.389997 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31980e99-10d5-4dff-a92b-5dff0dd19796 service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:25:01.400266 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31980e99-10d5-4dff-a92b-5dff0dd19796 service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:25:01.406328 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a3a99-1ec1-490d-aa64-345a8b9b3ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:25:01.406944 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a3a99-1ec1-490d-aa64-345a8b9b3ce9 admin admin] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:25:01.407228 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a3a99-1ec1-490d-aa64-345a8b9b3ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:25:01.407506 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a3a99-1ec1-490d-aa64-345a8b9b3ce9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:25:01.408145 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a3a99-1ec1-490d-aa64-345a8b9b3ce9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:25:01.410096 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a3a99-1ec1-490d-aa64-345a8b9b3ce9 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:25:01.444582 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1306/2604] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 00:25:01 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:25:01.448840 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39a385-e863-4d87-91eb-5dd4538ebe86 None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:25:01.461080 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39a385-e863-4d87-91eb-5dd4538ebe86 None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:25:01.536084 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abf5d4-f317-4275-952b-ea9632079c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:25:01.536752 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abf5d4-f317-4275-952b-ea9632079c0f admin admin] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:25:01.537134 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abf5d4-f317-4275-952b-ea9632079c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:25:01.537484 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abf5d4-f317-4275-952b-ea9632079c0f admin admin] PATH_INFO: `/v3/users` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:25:01.538196 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9abf5d4-f317-4275-952b-ea9632079c0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:25:01.540269 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9abf5d4-f317-4275-952b-ea9632079c0f admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:25:01.561588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:25:01.568519 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1299/2605] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 00:25:01 2022] POST /identity/v3/users => generated 296 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:25:01.572458 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a3a99-1ec1-490d-aa64-345a8b9b3ce9 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:25:01.583271 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a3a99-1ec1-490d-aa64-345a8b9b3ce9 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:25:01.589140 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b49a2b-162c-430e-accb-0c73c93dafa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:25:01.589855 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b49a2b-162c-430e-accb-0c73c93dafa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:25:01.590222 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b49a2b-162c-430e-accb-0c73c93dafa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:25:01.590506 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b49a2b-162c-430e-accb-0c73c93dafa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:25:01.591162 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b49a2b-162c-430e-accb-0c73c93dafa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:25:01.592809 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b49a2b-162c-430e-accb-0c73c93dafa9 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:25:01.605092 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1307/2606] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:25:01.609412 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abf5d4-f317-4275-952b-ea9632079c0f admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:25:01.623294 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abf5d4-f317-4275-952b-ea9632079c0f admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:25:01.629261 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff16c527-d40e-40fd-8aac-d42a72d5b3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:25:01.630065 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff16c527-d40e-40fd-8aac-d42a72d5b3f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:25:01.630454 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff16c527-d40e-40fd-8aac-d42a72d5b3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:25:01.630839 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff16c527-d40e-40fd-8aac-d42a72d5b3f6 admin admin] PATH_INFO: `/v3/projects/8a32c2203a304c62b0a88e512982a904/users/6652e4b4060a4d44bdce2f1bf5632bcc/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:25:01.650277 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff16c527-d40e-40fd-8aac-d42a72d5b3f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a32c2203a304c62b0a88e512982a904, user_id=6652e4b4060a4d44bdce2f1bf5632bcc, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:25:01.653952 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff16c527-d40e-40fd-8aac-d42a72d5b3f6 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:25:01.673333 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1300/2607] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:25:01 2022] PUT /identity/v3/projects/8a32c2203a304c62b0a88e512982a904/users/6652e4b4060a4d44bdce2f1bf5632bcc/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:25:01.677416 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b49a2b-162c-430e-accb-0c73c93dafa9 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:25:01.688438 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b49a2b-162c-430e-accb-0c73c93dafa9 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:25:01.715214 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:25:01.761077 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5903255-a0eb-454e-9b3f-b8accabf673e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:25:01.761959 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5903255-a0eb-454e-9b3f-b8accabf673e admin admin] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:25:01.762252 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5903255-a0eb-454e-9b3f-b8accabf673e admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:25:01.762531 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5903255-a0eb-454e-9b3f-b8accabf673e admin admin] PATH_INFO: `/v3/roles` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:25:01.763178 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5903255-a0eb-454e-9b3f-b8accabf673e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:25:01.764822 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5903255-a0eb-454e-9b3f-b8accabf673e admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:25:01.775052 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1308/2608] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 00:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:25:01.779199 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff16c527-d40e-40fd-8aac-d42a72d5b3f6 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:25:01.789746 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff16c527-d40e-40fd-8aac-d42a72d5b3f6 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:25:01.794569 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9e52f-9d56-427b-bda2-cb02ac3cd78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:25:01.795226 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9e52f-9d56-427b-bda2-cb02ac3cd78a admin admin] REQUEST_METHOD: `PUT` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:25:01.795512 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9e52f-9d56-427b-bda2-cb02ac3cd78a admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:25:01.795788 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d9e52f-9d56-427b-bda2-cb02ac3cd78a admin admin] PATH_INFO: `/v3/projects/8a32c2203a304c62b0a88e512982a904/users/6652e4b4060a4d44bdce2f1bf5632bcc/roles/839a55263e2c4cbaa2f945b75cf298d6` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:25:01.797723 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9e52f-9d56-427b-bda2-cb02ac3cd78a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a32c2203a304c62b0a88e512982a904, user_id=6652e4b4060a4d44bdce2f1bf5632bcc, role_id=839a55263e2c4cbaa2f945b75cf298d6)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:25:01.800328 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d9e52f-9d56-427b-bda2-cb02ac3cd78a admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:25:01.811658 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1301/2609] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 00:25:01 2022] PUT /identity/v3/projects/8a32c2203a304c62b0a88e512982a904/users/6652e4b4060a4d44bdce2f1bf5632bcc/roles/839a55263e2c4cbaa2f945b75cf298d6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:25:06.512982 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c6eac-0fd6-434f-b877-ab330e5dc12c None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:25:06.513451 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c6eac-0fd6-434f-b877-ab330e5dc12c None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:25:06.513570 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c6eac-0fd6-434f-b877-ab330e5dc12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:25:06.543567 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-bc4c6eac-0fd6-434f-b877-ab330e5dc12c None None] MFA Rules not processed for user `6652e4b4060a4d44bdce2f1bf5632bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:25:06.544821 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-bc4c6eac-0fd6-434f-b877-ab330e5dc12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:25:06.617591 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1309/2610] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:25:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:25:06.635190 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9e52f-9d56-427b-bda2-cb02ac3cd78a admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:25:06.638105 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-39d9e52f-9d56-427b-bda2-cb02ac3cd78a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:25:06.721170 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d9e52f-9d56-427b-bda2-cb02ac3cd78a admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:25:06.726823 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196b307-aa11-4dae-a3d4-b019084175e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:25:06.727436 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196b307-aa11-4dae-a3d4-b019084175e4 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:25:06.727721 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196b307-aa11-4dae-a3d4-b019084175e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:25:06.728001 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196b307-aa11-4dae-a3d4-b019084175e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:25:06.789283 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196b307-aa11-4dae-a3d4-b019084175e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:25:06.790865 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196b307-aa11-4dae-a3d4-b019084175e4 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:25:06.800923 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1302/2611] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:25:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:25:16.614514 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=54 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:25:16.796671 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=71,keystone:INSERT=2 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:25:30.450508 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04260f71-93a1-45cc-ab24-cefa2db6f6f0 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:25:30.451751 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04260f71-93a1-45cc-ab24-cefa2db6f6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:25:30.451751 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04260f71-93a1-45cc-ab24-cefa2db6f6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:25:30.470308 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-04260f71-93a1-45cc-ab24-cefa2db6f6f0 None None] MFA Rules not processed for user `163f3d7a305f4d8b8eb14794a51acda4`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:25:30.471500 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-04260f71-93a1-45cc-ab24-cefa2db6f6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:25:30.478234 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1310/2612] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:25:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:25:40.465374 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=3 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:25:46.518905 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196b307-aa11-4dae-a3d4-b019084175e4 service nova] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:25:46.573989 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196b307-aa11-4dae-a3d4-b019084175e4 service nova] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:25:46.577717 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec41414-b55c-4a7f-bc84-ab073d1b943b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:25:46.578320 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec41414-b55c-4a7f-bc84-ab073d1b943b service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:25:46.578605 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec41414-b55c-4a7f-bc84-ab073d1b943b service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:25:46.578886 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec41414-b55c-4a7f-bc84-ab073d1b943b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:25:46.587836 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec41414-b55c-4a7f-bc84-ab073d1b943b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:25:46.589284 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec41414-b55c-4a7f-bc84-ab073d1b943b service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:25:46.599555 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1303/2613] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:25:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:25:56.595138 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=26 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:27:15.838641 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04260f71-93a1-45cc-ab24-cefa2db6f6f0 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:27:15.840054 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-04260f71-93a1-45cc-ab24-cefa2db6f6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:27:15.976764 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04260f71-93a1-45cc-ab24-cefa2db6f6f0 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:27:15.981866 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece50aa-f99c-43a2-8c49-6f15946c5274 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24108b6355414f64ba5e7bf84011eee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:27:15.982474 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece50aa-f99c-43a2-8c49-6f15946c5274 service placement] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:27:15.982761 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece50aa-f99c-43a2-8c49-6f15946c5274 service placement] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:27:15.983041 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece50aa-f99c-43a2-8c49-6f15946c5274 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:27:15.984791 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-aece50aa-f99c-43a2-8c49-6f15946c5274 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:27:15.996359 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece50aa-f99c-43a2-8c49-6f15946c5274 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:27:15.997946 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece50aa-f99c-43a2-8c49-6f15946c5274 service placement] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:27:16.008507 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1311/2614] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:27:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:27:26.003767 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=63 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:28:07.060920 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5efa80-a77f-48c4-841b-0b3b4259e46b None None] REQUEST_METHOD: `POST` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:28:07.061410 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5efa80-a77f-48c4-841b-0b3b4259e46b None None] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:28:07.061511 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5efa80-a77f-48c4-841b-0b3b4259e46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:28:07.080211 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.auth.core [None req-1d5efa80-a77f-48c4-841b-0b3b4259e46b None None] MFA Rules not processed for user `163f3d7a305f4d8b8eb14794a51acda4`. Rule list: `[]` (Enabled: `True`). {{(pid=81381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:28:07.081449 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-1d5efa80-a77f-48c4-841b-0b3b4259e46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:28:07.088752 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1304/2615] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:28:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:28:07.404873 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece50aa-f99c-43a2-8c49-6f15946c5274 service placement] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:28:07.423869 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece50aa-f99c-43a2-8c49-6f15946c5274 service placement] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:28:07.427585 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5aeda-203b-4a23-a73b-87d9a3641251 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:28:07.429031 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5aeda-203b-4a23-a73b-87d9a3641251 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:28:07.429199 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5aeda-203b-4a23-a73b-87d9a3641251 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:28:07.429395 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5aeda-203b-4a23-a73b-87d9a3641251 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:28:07.431133 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b7f5aeda-203b-4a23-a73b-87d9a3641251 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:28:07.453227 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f5aeda-203b-4a23-a73b-87d9a3641251 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:28:07.454647 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f5aeda-203b-4a23-a73b-87d9a3641251 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:28:07.462876 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1312/2616] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:28:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:28:07.903463 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5efa80-a77f-48c4-841b-0b3b4259e46b None None] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:28:07.916998 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5efa80-a77f-48c4-841b-0b3b4259e46b None None] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:28:07.922408 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86e3fd-e19c-4b3c-b0a4-9cabf6047554 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24108b6355414f64ba5e7bf84011eee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:28:07.923025 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86e3fd-e19c-4b3c-b0a4-9cabf6047554 service placement] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:28:07.923310 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86e3fd-e19c-4b3c-b0a4-9cabf6047554 service placement] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:28:07.923585 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86e3fd-e19c-4b3c-b0a4-9cabf6047554 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:28:07.925349 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ec86e3fd-e19c-4b3c-b0a4-9cabf6047554 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:28:07.937421 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86e3fd-e19c-4b3c-b0a4-9cabf6047554 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:28:07.939015 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86e3fd-e19c-4b3c-b0a4-9cabf6047554 service placement] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:28:07.949528 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1305/2617] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:28:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:28:17.459060 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=15 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:28:17.945081 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=9 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:28:21.089481 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5aeda-203b-4a23-a73b-87d9a3641251 service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:28:21.119925 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5aeda-203b-4a23-a73b-87d9a3641251 service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:28:21.125830 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8d1b8-3e6a-45a9-8ed4-a940e5057fab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:28:21.126440 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8d1b8-3e6a-45a9-8ed4-a940e5057fab service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:28:21.126731 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8d1b8-3e6a-45a9-8ed4-a940e5057fab service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:28:21.127009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8d1b8-3e6a-45a9-8ed4-a940e5057fab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:28:21.129181 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9ba8d1b8-3e6a-45a9-8ed4-a940e5057fab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:28:21.156042 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba8d1b8-3e6a-45a9-8ed4-a940e5057fab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:28:21.157679 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba8d1b8-3e6a-45a9-8ed4-a940e5057fab service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:28:21.168407 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1313/2618] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:28:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:28:24.902059 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86e3fd-e19c-4b3c-b0a4-9cabf6047554 service placement] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:28:24.914463 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86e3fd-e19c-4b3c-b0a4-9cabf6047554 service placement] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:28:24.920884 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad1914-c116-4614-b9c5-d7bc81cd53d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:28:24.921489 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad1914-c116-4614-b9c5-d7bc81cd53d5 service nova] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:28:24.921769 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad1914-c116-4614-b9c5-d7bc81cd53d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:28:24.922044 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad1914-c116-4614-b9c5-d7bc81cd53d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:28:24.923792 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-56ad1914-c116-4614-b9c5-d7bc81cd53d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:28:24.936757 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad1914-c116-4614-b9c5-d7bc81cd53d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:28:24.938310 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad1914-c116-4614-b9c5-d7bc81cd53d5 service nova] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:28:24.949133 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1306/2619] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:28:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:28:31.163810 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=18 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:28:34.945251 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=6 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:30:13.873496 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8d1b8-3e6a-45a9-8ed4-a940e5057fab service nova] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:30:13.874974 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9ba8d1b8-3e6a-45a9-8ed4-a940e5057fab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:30:13.953392 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8d1b8-3e6a-45a9-8ed4-a940e5057fab service nova] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:30:13.956995 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1a441-8632-45d8-ac11-709a2a6e4d34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:30:13.957592 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1a441-8632-45d8-ac11-709a2a6e4d34 service cinder] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:30:13.957874 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1a441-8632-45d8-ac11-709a2a6e4d34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:30:13.958150 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1a441-8632-45d8-ac11-709a2a6e4d34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:30:13.966671 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1a441-8632-45d8-ac11-709a2a6e4d34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:30:13.968199 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1a441-8632-45d8-ac11-709a2a6e4d34 service cinder] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:30:13.976856 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1314/2620] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:30:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:30:21.573880 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad1914-c116-4614-b9c5-d7bc81cd53d5 service nova] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:30:21.575479 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-56ad1914-c116-4614-b9c5-d7bc81cd53d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:30:21.586711 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad1914-c116-4614-b9c5-d7bc81cd53d5 service nova] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:30:21.590722 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47724f-9b4b-4429-ad99-2ad040e5be83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:30:21.591309 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47724f-9b4b-4429-ad99-2ad040e5be83 service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:30:21.591588 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47724f-9b4b-4429-ad99-2ad040e5be83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:30:21.591862 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47724f-9b4b-4429-ad99-2ad040e5be83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:30:21.593580 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-af47724f-9b4b-4429-ad99-2ad040e5be83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:30:21.602681 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af47724f-9b4b-4429-ad99-2ad040e5be83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:30:21.604071 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af47724f-9b4b-4429-ad99-2ad040e5be83 service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:30:21.612862 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1307/2621] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:30:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:30:23.964032 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47d01e-e007-41b1-8dc3-560870f6dcc5 None None] REQUEST_METHOD: `POST` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:30:23.964609 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47d01e-e007-41b1-8dc3-560870f6dcc5 None None] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:30:23.964726 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47d01e-e007-41b1-8dc3-560870f6dcc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:30:23.983676 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.auth.core [None req-8a47d01e-e007-41b1-8dc3-560870f6dcc5 None None] MFA Rules not processed for user `163f3d7a305f4d8b8eb14794a51acda4`. Rule list: `[]` (Enabled: `True`). {{(pid=81380) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:30:23.984900 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-8a47d01e-e007-41b1-8dc3-560870f6dcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:30:23.991517 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1315/2622] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 00:30:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:30:24.036004 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47724f-9b4b-4429-ad99-2ad040e5be83 service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:30:24.037445 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-af47724f-9b4b-4429-ad99-2ad040e5be83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:30:24.048889 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47724f-9b4b-4429-ad99-2ad040e5be83 service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:30:24.052664 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0dbce-8b91-4424-9bff-77e744e270e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd170de478a344fc98951697c7a6298d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:30:24.053264 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0dbce-8b91-4424-9bff-77e744e270e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:30:24.053548 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0dbce-8b91-4424-9bff-77e744e270e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:30:24.053847 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0dbce-8b91-4424-9bff-77e744e270e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:30:24.055526 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-50b0dbce-8b91-4424-9bff-77e744e270e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:30:24.068862 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b0dbce-8b91-4424-9bff-77e744e270e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:30:24.070260 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b0dbce-8b91-4424-9bff-77e744e270e7 service cinder] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:30:24.081601 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1308/2623] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:30:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:30:24.368367 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47d01e-e007-41b1-8dc3-560870f6dcc5 None None] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:30:24.378776 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a47d01e-e007-41b1-8dc3-560870f6dcc5 None None] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:30:24.382539 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffea429-b3b5-47d7-9069-4e59adefab37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:30:24.383394 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffea429-b3b5-47d7-9069-4e59adefab37 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:30:24.383678 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffea429-b3b5-47d7-9069-4e59adefab37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:30:24.383957 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffea429-b3b5-47d7-9069-4e59adefab37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:30:24.385722 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-0ffea429-b3b5-47d7-9069-4e59adefab37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:30:24.395657 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffea429-b3b5-47d7-9069-4e59adefab37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:30:24.397115 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffea429-b3b5-47d7-9069-4e59adefab37 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:30:24.405103 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1316/2624] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:30:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:30:24.862386 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0dbce-8b91-4424-9bff-77e744e270e7 service cinder] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:30:24.871924 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0dbce-8b91-4424-9bff-77e744e270e7 service cinder] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:30:24.875980 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d265f-a593-4915-b8db-a5fccdeeca2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:30:24.876611 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d265f-a593-4915-b8db-a5fccdeeca2e service neutron] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:30:24.876891 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d265f-a593-4915-b8db-a5fccdeeca2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:30:24.877161 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d265f-a593-4915-b8db-a5fccdeeca2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:30:24.878893 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-ca0d265f-a593-4915-b8db-a5fccdeeca2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:30:24.893697 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0d265f-a593-4915-b8db-a5fccdeeca2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:30:24.895076 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0d265f-a593-4915-b8db-a5fccdeeca2e service neutron] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:30:24.906143 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1309/2625] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:30:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:30:27.445630 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffea429-b3b5-47d7-9069-4e59adefab37 service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:30:27.448148 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-0ffea429-b3b5-47d7-9069-4e59adefab37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:30:27.465685 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffea429-b3b5-47d7-9069-4e59adefab37 service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:30:27.471662 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bb73d-43ff-47a5-915e-1dc1cc5174d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c4a3579d56294b9d8ad36d4d174c8526', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:30:27.472370 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bb73d-43ff-47a5-915e-1dc1cc5174d6 service nova] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:30:27.472752 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bb73d-43ff-47a5-915e-1dc1cc5174d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:30:27.473097 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bb73d-43ff-47a5-915e-1dc1cc5174d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:30:27.474972 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-9e3bb73d-43ff-47a5-915e-1dc1cc5174d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:30:27.489122 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3bb73d-43ff-47a5-915e-1dc1cc5174d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:30:27.490782 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3bb73d-43ff-47a5-915e-1dc1cc5174d6 service nova] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:30:27.502477 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1317/2626] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:30:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:30:34.901013 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=18 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:30:37.499349 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=50 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:32:20.852934 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d265f-a593-4915-b8db-a5fccdeeca2e service neutron] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:32:20.864479 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d265f-a593-4915-b8db-a5fccdeeca2e service neutron] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:32:20.869819 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b9625-1893-421c-b630-1b45ff1b9fe8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24108b6355414f64ba5e7bf84011eee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:32:20.870385 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b9625-1893-421c-b630-1b45ff1b9fe8 service placement] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:32:20.870661 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b9625-1893-421c-b630-1b45ff1b9fe8 service placement] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:32:20.870935 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b9625-1893-421c-b630-1b45ff1b9fe8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:32:20.881050 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b9625-1893-421c-b630-1b45ff1b9fe8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:32:20.882626 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b9625-1893-421c-b630-1b45ff1b9fe8 service placement] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:32:20.893152 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1310/2627] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:32:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:32:30.888747 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=6 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:32:46.692009 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bb73d-43ff-47a5-915e-1dc1cc5174d6 service nova] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:32:46.701741 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bb73d-43ff-47a5-915e-1dc1cc5174d6 service nova] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:32:46.705396 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12730cd4-6e77-4004-bc5d-46e7f7eeefe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a09d45d080bc461c88f38ff281b75b06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:32:46.706026 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12730cd4-6e77-4004-bc5d-46e7f7eeefe3 service neutron] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:32:46.706308 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12730cd4-6e77-4004-bc5d-46e7f7eeefe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:32:46.706585 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12730cd4-6e77-4004-bc5d-46e7f7eeefe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:32:46.781871 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12730cd4-6e77-4004-bc5d-46e7f7eeefe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:32:46.783198 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12730cd4-6e77-4004-bc5d-46e7f7eeefe3 service neutron] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:32:46.802624 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1318/2628] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:32:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:32:49.929991 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b9625-1893-421c-b630-1b45ff1b9fe8 service placement] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:32:49.932433 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-2a2b9625-1893-421c-b630-1b45ff1b9fe8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:32:49.957216 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b9625-1893-421c-b630-1b45ff1b9fe8 service placement] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:32:49.963334 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aaaac0-fb26-4712-9e30-a82353aac717 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24108b6355414f64ba5e7bf84011eee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:32:49.964024 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aaaac0-fb26-4712-9e30-a82353aac717 service placement] REQUEST_METHOD: `GET` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:32:49.964600 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aaaac0-fb26-4712-9e30-a82353aac717 service placement] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:32:49.965112 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aaaac0-fb26-4712-9e30-a82353aac717 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:32:49.969209 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-69aaaac0-fb26-4712-9e30-a82353aac717 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:32:49.983214 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aaaac0-fb26-4712-9e30-a82353aac717 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:32:49.987917 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aaaac0-fb26-4712-9e30-a82353aac717 service placement] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:32:49.996667 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1311/2629] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:32:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:32:53.153195 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12730cd4-6e77-4004-bc5d-46e7f7eeefe3 service neutron] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:32:53.163960 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12730cd4-6e77-4004-bc5d-46e7f7eeefe3 service neutron] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:32:53.174220 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90434a6-4f9a-477e-a645-dd483a36af1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:32:53.174859 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90434a6-4f9a-477e-a645-dd483a36af1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:32:53.175138 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90434a6-4f9a-477e-a645-dd483a36af1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:32:53.175414 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90434a6-4f9a-477e-a645-dd483a36af1e admin admin] PATH_INFO: `/v3/users/163f3d7a305f4d8b8eb14794a51acda4` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:32:53.176557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90434a6-4f9a-477e-a645-dd483a36af1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=163f3d7a305f4d8b8eb14794a51acda4)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:32:53.179629 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90434a6-4f9a-477e-a645-dd483a36af1e admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:32:53.226882 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-f90434a6-4f9a-477e-a645-dd483a36af1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '163f3d7a305f4d8b8eb14794a51acda4'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:32:53.227521 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-f90434a6-4f9a-477e-a645-dd483a36af1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '163f3d7a305f4d8b8eb14794a51acda4'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:32:53.234589 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-f90434a6-4f9a-477e-a645-dd483a36af1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '163f3d7a305f4d8b8eb14794a51acda4'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:32:53.246487 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-f90434a6-4f9a-477e-a645-dd483a36af1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '163f3d7a305f4d8b8eb14794a51acda4'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:32:53.253711 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-f90434a6-4f9a-477e-a645-dd483a36af1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '163f3d7a305f4d8b8eb14794a51acda4'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:32:53.259890 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1319/2630] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:32:53 2022] DELETE /identity/v3/users/163f3d7a305f4d8b8eb14794a51acda4 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:32:53.431989 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aaaac0-fb26-4712-9e30-a82353aac717 service placement] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:32:53.442451 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aaaac0-fb26-4712-9e30-a82353aac717 service placement] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:32:53.498839 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55370331-01cc-446c-826e-1ed731c4f212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:32:53.499466 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55370331-01cc-446c-826e-1ed731c4f212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:32:53.499750 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55370331-01cc-446c-826e-1ed731c4f212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:32:53.500028 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55370331-01cc-446c-826e-1ed731c4f212 admin admin] PATH_INFO: `/v3/projects/4a4117126a8d4004a3878e10554b0d5c` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:32:53.501158 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55370331-01cc-446c-826e-1ed731c4f212 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a4117126a8d4004a3878e10554b0d5c)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:32:53.503026 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55370331-01cc-446c-826e-1ed731c4f212 admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:32:53.549418 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-55370331-01cc-446c-826e-1ed731c4f212 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a4117126a8d4004a3878e10554b0d5c', 'request_id': 'req-55370331-01cc-446c-826e-1ed731c4f212'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:32:53.555778 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-55370331-01cc-446c-826e-1ed731c4f212 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a4117126a8d4004a3878e10554b0d5c', 'request_id': 'req-55370331-01cc-446c-826e-1ed731c4f212'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:32:53.566806 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-55370331-01cc-446c-826e-1ed731c4f212 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a4117126a8d4004a3878e10554b0d5c', 'request_id': 'req-55370331-01cc-446c-826e-1ed731c4f212'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:32:53.590926 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-55370331-01cc-446c-826e-1ed731c4f212 admin admin] The token cache is being invalidate because project 4a4117126a8d4004a3878e10554b0d5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:32:53.591223 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-55370331-01cc-446c-826e-1ed731c4f212 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:32:53.591848 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-55370331-01cc-446c-826e-1ed731c4f212 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:32:53.593423 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1312/2631] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:32:53 2022] DELETE /identity/v3/projects/4a4117126a8d4004a3878e10554b0d5c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:32:53.597557 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90434a6-4f9a-477e-a645-dd483a36af1e admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:32:53.598978 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-f90434a6-4f9a-477e-a645-dd483a36af1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:32:53.611756 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90434a6-4f9a-477e-a645-dd483a36af1e admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:32:53.655715 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:32:53.668602 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271daaf-36eb-45d9-8d87-28c9f03ae307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:32:53.669229 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271daaf-36eb-45d9-8d87-28c9f03ae307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:32:53.669514 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271daaf-36eb-45d9-8d87-28c9f03ae307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:32:53.669790 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271daaf-36eb-45d9-8d87-28c9f03ae307 admin admin] PATH_INFO: `/v3/users/6652e4b4060a4d44bdce2f1bf5632bcc` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:32:53.670909 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271daaf-36eb-45d9-8d87-28c9f03ae307 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6652e4b4060a4d44bdce2f1bf5632bcc)` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:32:53.672847 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271daaf-36eb-45d9-8d87-28c9f03ae307 admin admin] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:32:53.718757 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b271daaf-36eb-45d9-8d87-28c9f03ae307 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6652e4b4060a4d44bdce2f1bf5632bcc'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:32:53.719410 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b271daaf-36eb-45d9-8d87-28c9f03ae307 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6652e4b4060a4d44bdce2f1bf5632bcc'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:32:53.726813 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b271daaf-36eb-45d9-8d87-28c9f03ae307 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6652e4b4060a4d44bdce2f1bf5632bcc'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:32:53.738370 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b271daaf-36eb-45d9-8d87-28c9f03ae307 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6652e4b4060a4d44bdce2f1bf5632bcc'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:32:53.745892 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.notifications [None req-b271daaf-36eb-45d9-8d87-28c9f03ae307 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6652e4b4060a4d44bdce2f1bf5632bcc'} {{(pid=81380) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:32:53.751884 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1320/2632] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 00:32:53 2022] DELETE /identity/v3/users/6652e4b4060a4d44bdce2f1bf5632bcc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:32:53.923381 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55370331-01cc-446c-826e-1ed731c4f212 admin admin] Authenticating user token {{(pid=81381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:32:53.925184 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.fernet_utils [None req-55370331-01cc-446c-826e-1ed731c4f212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:32:53.939564 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55370331-01cc-446c-826e-1ed731c4f212 admin admin] Validating token access rules against request {{(pid=81381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:32:53.999142 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9405dc4-2df0-4e95-81a1-76d1b56effdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac8d80b2c4a24dfea25c6de809bc37e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e646fb7093394a37b996a9f2520c5910', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:32:53.999784 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9405dc4-2df0-4e95-81a1-76d1b56effdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:32:54.000070 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9405dc4-2df0-4e95-81a1-76d1b56effdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:32:54.000349 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9405dc4-2df0-4e95-81a1-76d1b56effdf admin admin] PATH_INFO: `/v3/projects/8a32c2203a304c62b0a88e512982a904` {{(pid=81381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:32:54.001501 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9405dc4-2df0-4e95-81a1-76d1b56effdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a32c2203a304c62b0a88e512982a904)` {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:32:54.003375 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9405dc4-2df0-4e95-81a1-76d1b56effdf admin admin] RBAC: Authorization granted {{(pid=81381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:32:54.011307 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:32:54.052073 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d9405dc4-2df0-4e95-81a1-76d1b56effdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a32c2203a304c62b0a88e512982a904', 'request_id': 'req-d9405dc4-2df0-4e95-81a1-76d1b56effdf'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:32:54.058554 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d9405dc4-2df0-4e95-81a1-76d1b56effdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a32c2203a304c62b0a88e512982a904', 'request_id': 'req-d9405dc4-2df0-4e95-81a1-76d1b56effdf'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:32:54.065795 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d9405dc4-2df0-4e95-81a1-76d1b56effdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a32c2203a304c62b0a88e512982a904', 'request_id': 'req-d9405dc4-2df0-4e95-81a1-76d1b56effdf'} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:32:54.076374 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d9405dc4-2df0-4e95-81a1-76d1b56effdf admin admin] The token cache is being invalidate because project 8a32c2203a304c62b0a88e512982a904 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:32:54.076699 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d9405dc4-2df0-4e95-81a1-76d1b56effdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:32:54.077338 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG keystone.notifications [None req-d9405dc4-2df0-4e95-81a1-76d1b56effdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:32:54.078843 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: [pid: 81381|app: 0|req: 1313/2633] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 00:32:53 2022] DELETE /identity/v3/projects/8a32c2203a304c62b0a88e512982a904 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:33:03.748247 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:33:04.073247 dv-cinder-861566-5-iscsi devstack@keystone.service[81381]: DEBUG dbcounter [-] [81381] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:37:24.802961 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271daaf-36eb-45d9-8d87-28c9f03ae307 admin admin] Authenticating user token {{(pid=81380) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:37:24.804372 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-b271daaf-36eb-45d9-8d87-28c9f03ae307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:37:24.941082 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271daaf-36eb-45d9-8d87-28c9f03ae307 admin admin] Validating token access rules against request {{(pid=81380) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:37:24.997546 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa7db2-c3c1-4fc4-98b2-a5868d87ee94 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24108b6355414f64ba5e7bf84011eee6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c9246c059754504afee4711d113a3fe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81380) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:37:24.998165 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa7db2-c3c1-4fc4-98b2-a5868d87ee94 service placement] REQUEST_METHOD: `GET` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:37:24.998436 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa7db2-c3c1-4fc4-98b2-a5868d87ee94 service placement] SCRIPT_NAME: `/identity` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:37:24.998709 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa7db2-c3c1-4fc4-98b2-a5868d87ee94 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81380) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:37:25.000428 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.fernet_utils [None req-4efa7db2-c3c1-4fc4-98b2-a5868d87ee94 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81380) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:37:25.011719 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efa7db2-c3c1-4fc4-98b2-a5868d87ee94 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:37:25.013311 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efa7db2-c3c1-4fc4-98b2-a5868d87ee94 service placement] RBAC: Authorization granted {{(pid=81380) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:37:25.023143 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: [pid: 81380|app: 0|req: 1321/2634] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 00:37:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:37:35.019730 dv-cinder-861566-5-iscsi devstack@keystone.service[81380]: DEBUG dbcounter [-] [81380] Writing DB stats keystone:SELECT=89 {{(pid=81380) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}